Noise-trained deep neural networks effectively predict human vision and its neural responses to challenging images.

Deep neural networks (DNNs) for object classification have been argued to provide the most promising model of the visual system, accompanied by claims that they have attained or even surpassed human-level performance. Here, we evaluated whether DNNs provide a viable model of human vision when tested with challenging noisy images of objects, sometimes presented at the very limits of visibility. We show that popular state-of-the-art DNNs perform in a qualitatively different manner than humans-they are unusually susceptible to spatially uncorrelated white noise and less impaired by spatially correlated noise. We implemented a noise training procedure to determine whether noise-trained DNNs exhibit more robust responses that better match human behavioral and neural performance. We found that noise-trained DNNs provide a better qualitative match to human performance; moreover, they reliably predict human recognition thresholds on an image-by-image basis. Functional neuroimaging revealed that noise-trained DNNs provide a better correspondence to the pattern-specific neural representations found in both early visual areas and high-level object areas. A layer-specific analysis of the DNNs indicated that noise training led to broad-ranging modifications throughout the network, with greater benefits of noise robustness accruing in progressively higher layers. Our findings demonstrate that noise-trained DNNs provide a viable model to account for human behavioral and neural responses to objects in challenging noisy viewing conditions. Further, they suggest that robustness to noise may be acquired through a process of visual learning.

Spikiness and animacy as potential organizing principles of human ventral visual cortex.

Considerable research has been devoted to understanding the fundamental organizing principles of the ventral visual pathway. A recent study revealed a series of 3-4 topographical maps arranged along the macaque inferotemporal (IT) cortex. The maps articulated a two-dimensional space based on the spikiness and animacy of visual objects, with "inanimate-spiky" and "inanimate-stubby" regions of the maps constituting two previously unidentified cortical networks. The goal of our study was to determine whether a similar functional organization might exist in human IT. To address this question, we presented the same object stimuli and images from "classic" object categories (bodies, faces, houses) to humans while recording fMRI activity at 7 Tesla. Contrasts designed to reveal the spikiness-animacy object space evoked extensive significant activation across human IT. However, unlike the macaque, we did not observe a clear sequence of complete maps, and selectivity for the spikiness-animacy space was deeply and mutually entangled with category-selectivity. Instead, we observed multiple new stimulus preferences in category-selective regions, including functional sub-structure related to object spikiness in scene-selective cortex. Taken together, these findings highlight spikiness as a promising organizing principle of human IT and provide new insights into the role of category-selective regions in visual object processing.

Convolutional neural network models applied to neuronal responses in macaque V1 reveal limited nonlinear processing.

Computational models of the primary visual cortex (V1) have suggested that V1 neurons behave like Gabor filters followed by simple nonlinearities. However, recent work employing convolutional neural network (CNN) models has suggested that V1 relies on far more nonlinear computations than previously thought. Specifically, unit responses in an intermediate layer of VGG-19 were found to best predict macaque V1 responses to thousands of natural and synthetic images. Here, we evaluated the hypothesis that the poor performance of lower layer units in VGG-19 might be attributable to their small receptive field size rather than to their lack of complexity per se. We compared VGG-19 with AlexNet, which has much larger receptive fields in its lower layers. Whereas the best-performing layer of VGG-19 occurred after seven nonlinear steps, the first convolutional layer of AlexNet best predicted V1 responses. Although the predictive accuracy of VGG-19 was somewhat better than that of standard AlexNet, we found that a modified version of AlexNet could match the performance of VGG-19 after only a few nonlinear computations. Control analyses revealed that decreasing the size of the input images caused the best-performing layer of VGG-19 to shift to a lower layer, consistent with the hypothesis that the relationship between image size and receptive field size can strongly affect model performance. We conducted additional analyses using a Gabor pyramid model to test for nonlinear contributions of normalization and contrast saturation. Overall, our findings suggest that the feedforward responses of V1 neurons can be well explained by assuming only a few nonlinear processing stages.

Rapid adaptation of primate LGN neurons to drifting grating stimulation.

The visual system needs to dynamically adapt to changing environments. Much is known about the adaptive effects of constant stimulation over prolonged periods. However, there are open questions regarding adaptation to stimuli that are changing over time, interrupted, or repeated. Feature-specific adaptation to repeating stimuli has been shown to occur as early as primary visual cortex (V1), but there is also evidence for more generalized, fatigue-like adaptation that might occur at an earlier stage of processing. Here, we show adaptation in the lateral geniculate nucleus (LGN) of awake, fixating monkeys following brief (1 s) exposure to repeated cycles of a 4-Hz drifting grating. We examined the relative change of each neuron's response across successive (repeated) grating cycles. We found that neurons from all cell classes (parvocellular, magnocellular, and koniocellular) showed significant adaptation. However, only magnocellular neurons showed adaptation when responses were averaged to a population response. In contrast to firing rates, response variability was largely unaffected. Finally, adaptation was comparable between monocular and binocular stimulation, suggesting that rapid LGN adaptation is monocular in nature.NEW & NOTEWORTHY Neural adaptation can be defined as reduction of spiking responses following repeated or prolonged stimulation. Adaptation helps adjust neural responsiveness to avoid saturation and has been suggested to improve perceptual selectivity, information transmission, and predictive coding. Here, we report rapid adaptation to repeated cycles of gratings drifting over the receptive field of neurons at the earliest site of postretinal processing, the lateral geniculate nucleus of the thalamus.

Diffuse Angiogram-Negative Subarachnoid Hemorrhage is Associated with an Intermediate Clinical Course.

BACKGROUND: The cerebral angiography result is negative for an underlying vascular lesion in 15-20% of patients with nontraumatic subarachnoid hemorrhage (SAH). Patients with angiogram-negative SAH include those with perimesencephalic SAH and diffuse SAH. Consensus suggests that perimesencephalic SAH confers a more favorable prognosis than diffuse SAH. Limited data exist to contextualize the clinical course and prognosis of diffuse SAH in relation to aneurysmal SAH in terms of critical care complications, neurologic complications, and functional outcomes. Here we compare the clinical course and functional outcomes of patients with perimesencephalic SAH, diffuse SAH, and aneurysmal SAH to better characterize the prognostic implications of each SAH subtype. METHODS: We conducted a retrospective cohort study that included all patients with nontraumatic SAH admitted to a tertiary care referral center between January 1, 2012, and December 31, 2017. Bleed patterns were radiographically adjudicated, and patients were assigned to three groups: perimesencephalic SAH, diffuse SAH, and aneurysmal SAH. Patient demographics, complications, and clinical outcomes were reported and compared. RESULTS: Eighty-six patients with perimesencephalic SAH, 174 with diffuse SAH, and 998 with aneurysmal SAH presented during the study period. Patients with aneurysmal SAH were significantly more likely to be female, White, and active smokers. There were no significant differences between patients with diffuse SAH and perimesencephalic SAH patterns. Critical care complications were compared across all three groups, with significant between-group differences in hypotension and shock (3.5% vs. 16.1% vs. 38.4% for perimesencephalic SAH vs. diffuse SAH vs. aneurysmal SAH, respectively; p < 0.01) and endotracheal intubation (0% vs. 26.4% vs. 48.8% for perimesencephalic SAH vs. diffuse SAH vs. aneurysmal SAH, respectively; p < 0.01). Similar trends were noted with long-term supportive care with tracheostomy and gastrostomy tubes and length of stay. Cerebrospinal fluid diversion was increasingly required across bleed types (9.3% vs. 54.6% vs. 76.3% for perimesencephalic SAH vs. diffuse SAH vs. aneurysmal SAH, respectively, p < 0.001). Vasospasm and delayed cerebral ischemia were comparable between perimesencephalic SAH and diffuse SAH but significantly lower than aneurysmal SAH. Patients with diffuse SAH had intermediate functional outcomes, with significant rates of nonhome discharge (23.0%) and poor functional status on discharge (26.4%), significantly higher than patients with perimesencephalic SAH and lower than patients with aneurysmal SAH. Diffuse SAH similarly conferred an intermediate rate of good functional outcomes at 1-6 months post discharge (92.3% vs. 78.6% vs. 47.3% for perimesencephalic SAH vs. diffuse SAH vs. aneurysmal SAH, respectively; p < 0.016). CONCLUSIONS: We confirm the consensus data that perimesencephalic SAH is associated with a more benign clinical course but demonstrate that diffuse SAH confers an intermediate prognosis, more malignant than perimesencephalic SAH but not as morbid as aneurysmal SAH. These results highlight the significant morbidity associated with diffuse SAH and emphasize need for vigilance in the acute care of these patients. These patients will likely benefit from continued high-acuity observation and potential support to avert significant risk of morbidity and neurologic compromise.

Resolving the Spatial Profile of Figure Enhancement in Human V1 through Population Receptive Field Modeling.

The detection and segmentation of meaningful figures from their background is one of the primary functions of vision. While work in nonhuman primates has implicated early visual mechanisms in this figure-ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7 Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex (N = 6, four females). We used a novel population receptive field mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure-ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually driven feedback from higher visual areas.SIGNIFICANCE STATEMENT A core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figure and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure-ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent with a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.

Convolutional neural networks trained with a developmental sequence of blurry to clear images reveal core differences between face and object processing.

Although convolutional neural networks (CNNs) provide a promising model for understanding human vision, most CNNs lack robustness to challenging viewing conditions, such as image blur, whereas human vision is much more reliable. Might robustness to blur be attributable to vision during infancy, given that acuity is initially poor but improves considerably over the first several months of life? Here, we evaluated the potential consequences of such early experiences by training CNN models on face and object recognition tasks while gradually reducing the amount of blur applied to the training images. For CNNs trained on blurry to clear faces, we observed sustained robustness to blur, consistent with a recent report by Vogelsang and colleagues (2018). By contrast, CNNs trained with blurry to clear objects failed to retain robustness to blur. Further analyses revealed that the spatial frequency tuning of the two CNNs was profoundly different. The blurry to clear face-trained network successfully retained a preference for low spatial frequencies, whereas the blurry to clear object-trained CNN exhibited a progressive shift toward higher spatial frequencies. Our findings provide novel computational evidence showing how face recognition, unlike object recognition, allows for more holistic processing. Moreover, our results suggest that blurry vision during infancy is insufficient to account for the robustness of adult vision to blurry objects.

Visual expectations change subjective experience without changing performance.

It is widely believed that visual expectations can change the subjective experiences of humans. We investigated how visual expectations in a recognition task affected objective performance and subjective perception. Using a 2-alternative-forced-choice task based on digit recognition of briefly presented and visually masked digits, we found over two experiments that expectations changed the quality of the experiences without changing the performance capabilities associated with the quality of experience. Expectations were manipulated by providing a cue indicating the set of possible digits that might appear on each trial. The results also inform the debate about whether subjective experiences can be categorized in a dichotomous manner or in a graded manner. We found that subjective experiences were graded near the objective threshold and more dichotomous away from the threshold. Furthermore, distinct expectations resulted in a more dichotomous distribution of subjective experience. We also provide evidence of an interesting relationship between stimulus duration, objective performance and subjective ratings. Only experiences that were rated as evoking some degree of perception showed systematic improvements in objective performance as a function of stimulus duration. These findings suggest that subjective experience cannot be understood without considering the broader cognitive context, namely that the quality of subjective experiences is dependent on a multitude of factors such as attention, task requirements and cognitive expectations.

Combined reconstructive and deconstructive endovascular approach for bilateral vertebral artery dissection with subarachnoid hemorrhage.

The video highlights a challenging case of bilateral vertebral artery dissection presenting with subarachnoid hemorrhage. The patient was found to have a critical flow-limiting stenosis in his dominant right vertebral artery and a ruptured pseudoaneurysm in his left vertebral artery. A single-stage endovascular treatment with stent reconstruction of the right vertebral artery and coil embolization sacrifice of the left side was performed. The case highlights the rationale for treatment and potential alternative strategies.The video can be found here: https://youtu.be/e0U_JE2jISw.

Representational Dynamics of Facial Viewpoint Encoding.

Faces provide a wealth of information, including the identity of the seen person and social cues, such as the direction of gaze. Crucially, different aspects of face processing require distinct forms of information encoding. Another person's attentional focus can be derived based on a view-dependent code. In contrast, identification benefits from invariance across all viewpoints. Different cortical areas have been suggested to subserve these distinct functions. However, little is known about the temporal aspects of differential viewpoint encoding in the human brain. Here, we combine EEG with multivariate data analyses to resolve the dynamics of face processing with high temporal resolution. This revealed a distinct sequence of viewpoint encoding. Head orientations were encoded first, starting after around 60 msec of processing. Shortly afterward, peaking around 115 msec after stimulus onset, a different encoding scheme emerged. At this latency, mirror-symmetric viewing angles elicited highly similar cortical responses. Finally, about 280 msec after visual onset, EEG response patterns demonstrated a considerable degree of viewpoint invariance across all viewpoints tested, with the noteworthy exception of the front-facing view. Taken together, our results indicate that the processing of facial viewpoints follows a temporal sequence of encoding schemes, potentially mirroring different levels of computational complexity.

Characterizing the effects of feature salience and top-down attention in the early visual system.

The visual system employs a sophisticated balance of attentional mechanisms: salient stimuli are prioritized for visual processing, yet observers can also ignore such stimuli when their goals require directing attention elsewhere. A powerful determinant of visual salience is local feature contrast: if a local region differs from its immediate surround along one or more feature dimensions, it will appear more salient. We used high-resolution functional MRI (fMRI) at 7T to characterize the modulatory effects of bottom-up salience and top-down voluntary attention within multiple sites along the early visual pathway, including visual areas V1-V4 and the lateral geniculate nucleus (LGN). Observers viewed arrays of spatially distributed gratings, where one of the gratings immediately to the left or right of fixation differed from all other items in orientation or motion direction, making it salient. To investigate the effects of directed attention, observers were cued to attend to the grating to the left or right of fixation, which was either salient or nonsalient. Results revealed reliable additive effects of top-down attention and stimulus-driven salience throughout visual areas V1-hV4. In comparison, the LGN exhibited significant attentional enhancement but was not reliably modulated by orientation- or motion-defined salience. Our findings indicate that top-down effects of spatial attention can influence visual processing at the earliest possible site along the visual pathway, including the LGN, whereas the processing of orientation- and motion-driven salience primarily involves feature-selective interactions that take place in early cortical visual areas.NEW & NOTEWORTHY While spatial attention allows for specific, goal-driven enhancement of stimuli, salient items outside of the current focus of attention must also be prioritized. We used 7T fMRI to compare salience and spatial attentional enhancement along the early visual hierarchy. We report additive effects of attention and bottom-up salience in early visual areas, suggesting that salience enhancement is not contingent on the observer's attentional state.

Reprioritization of Features of Multidimensional Objects Stored in Visual Working Memory.

A prevalent view of visual working memory (VWM) is that visual information is actively maintained in the form of perceptually integrated objects. Such reliance on object-based representations would predict that after an object is fully encoded into VWM, all features of that object would need to be maintained as a coherent unit. Here, we evaluated this idea by testing whether memory resources can be redeployed to a specific feature of an object already stored in VWM. We found that observers can utilize a retrospective cue presented during the maintenance period to attenuate both the gradual deterioration and complete loss of memory for a cued feature over time, but at the cost of accelerated loss of information regarding the uncued feature. Our findings demonstrate that object representations held within VWM can be decomposed into individual features and that having to retain additional features imposes greater demands on active maintenance processes.

Neural representation of form-contingent color filling-in in the early visual cortex.

Perceptual filling-in exemplifies the constructive nature of visual processing. Color, a prominent surface property of visual objects, can appear to spread to neighboring areas that lack any color. We investigated cortical responses to a color filling-in illusion that effectively dissociates perceived color from the retinal input (van Lier, Vergeer, & Anstis, 2009). Observers adapted to a star-shaped stimulus with alternating red- and cyan-colored points to elicit a complementary afterimage. By presenting an achromatic outline that enclosed one of the two afterimage colors, perceptual filling-in of that color was induced in the unadapted central region. Visual cortical activity was monitored with fMRI, and analyzed using multivariate pattern analysis. Activity patterns in early visual areas (V1-V4) reliably distinguished between the two color-induced filled-in conditions, but only higher extrastriate visual areas showed the predicted correspondence with color perception. Activity patterns allowed for reliable generalization between filled-in colors and physical presentations of perceptually matched colors in areas V3 and V4, but not in earlier visual areas. These findings suggest that the perception of filled-in surface color likely requires more extensive processing by extrastriate visual areas, in order for the neural representation of surface color to become aligned with perceptually matched real colors.

The Occipital Face Area Is Causally Involved in Facial Viewpoint Perception.

Humans reliably recognize faces across a range of viewpoints, but the neural substrates supporting this ability remain unclear. Recent work suggests that neural selectivity to mirror-symmetric viewpoints of faces, found across a large network of visual areas, may constitute a key computational step in achieving full viewpoint invariance. In this study, we used repetitive transcranial magnetic stimulation (rTMS) to test the hypothesis that the occipital face area (OFA), putatively a key node in the face network, plays a causal role in face viewpoint symmetry perception. Each participant underwent both offline rTMS to the right OFA and sham stimulation, preceding blocks of behavioral trials. After each stimulation period, the participant performed one of two behavioral tasks involving presentation of faces in the peripheral visual field: (1) judging the viewpoint symmetry; or (2) judging the angular rotation. rTMS applied to the right OFA significantly impaired performance in both tasks when stimuli were presented in the contralateral, left visual field. Interestingly, however, rTMS had a differential effect on the two tasks performed ipsilaterally. Although viewpoint symmetry judgments were significantly disrupted, we observed no effect on the angle judgment task. This interaction, caused by ipsilateral rTMS, provides support for models emphasizing the role of interhemispheric crosstalk in the formation of viewpoint-invariant face perception. SIGNIFICANCE STATEMENT: Faces are among the most salient objects we encounter during our everyday activities. Moreover, we are remarkably adept at identifying people at a glance, despite the diversity of viewpoints during our social encounters. Here, we investigate the cortical mechanisms underlying this ability by focusing on effects of viewpoint symmetry, i.e., the invariance of neural responses to mirror-symmetric facial viewpoints. We did this by temporarily disrupting neural processing in the occipital face area (OFA) using transcranial magnetic stimulation. Our results demonstrate that the OFA causally contributes to judgments facial viewpoints and suggest that effects of viewpoint symmetry, previously observed using fMRI, arise from an interhemispheric integration of visual information even when only one hemisphere receives direct visual stimulation.

Radial bias is not necessary for orientation decoding.

Multivariate pattern analysis can be used to decode the orientation of a viewed grating from fMRI signals in early visual areas. Although some studies have reported identifying multiple sources of the orientation information that make decoding possible, a recent study argued that orientation decoding is only possible because of a single source: a coarse-scale retinotopically organized preference for radial orientations. Here we aim to resolve these discrepant findings. We show that there were subtle, but critical, experimental design choices that led to the erroneous conclusion that a radial bias is the only source of orientation information in fMRI signals. In particular, we show that the reliance on a fast temporal-encoding paradigm for spatial mapping can be problematic, as effects of space and time become conflated and lead to distorted estimates of a voxel's orientation or retinotopic preference. When we implement minor changes to the temporal paradigm or to the visual stimulus itself, by slowing the periodic rotation of the stimulus or by smoothing its contrast-energy profile, we find significant evidence of orientation information that does not originate from radial bias. In an additional block-paradigm experiment where space and time were not conflated, we apply a formal model comparison approach and find that many voxels exhibit more complex tuning properties than predicted by radial bias alone or in combination with other known coarse-scale biases. Our findings support the conclusion that radial bias is not necessary for orientation decoding. In addition, our study highlights potential limitations of using temporal phase-encoded fMRI designs for characterizing voxel tuning properties.

Neural mechanisms of object-based attention.

What neural mechanisms underlie the ability to attend to a complex object in the presence of competing overlapping stimuli? We evaluated whether object-based attention might involve pattern-specific feedback to early visual areas to selectively enhance the set of low-level features corresponding to the attended object. Using fMRI and multivariate pattern analysis, we found that activity patterns in early visual areas (V1-V4) are strongly biased in favor of the attended object. Activity patterns evoked by single faces and single houses reliably predicted which of the 2 overlapping stimulus types was being attended with high accuracy (80-90% correct). Superior knowledge of upright objects led to improved attentional selection in early areas. Across individual blocks, the strength of the attentional bias signal in early visual areas was highly predictive of the modulations found in high-level object areas, implying that pattern-specific attentional filtering at early sites can determine the quality of object-specific signals that reach higher level visual areas. Through computational modeling, we show how feedback of an average template to V1-like units can improve discrimination of exemplars belonging to the attended category. Our findings provide a mechanistic account of how feedback to early visual areas can contribute to the attentional selection of complex objects.

Urokinase-type plasminogen activator promotes dendritic spine recovery and improves neurological outcome following ischemic stroke.

Spines are dendritic protrusions that receive most of the excitatory input in the brain. Early after the onset of cerebral ischemia dendritic spines in the peri-infarct cortex are replaced by areas of focal swelling, and their re-emergence from these varicosities is associated with neurological recovery after acute ischemic stroke (AIS). Urokinase-type plasminogen activator (uPA) is a serine proteinase that plays a central role in tissue remodeling via binding to the urokinase plasminogen activator receptor (uPAR). We report that cerebral cortical neurons release uPA during the recovery phase from ischemic stroke in vivo or hypoxia in vitro. Although uPA does not have an effect on ischemia- or hypoxia-induced neuronal death, genetic deficiency of uPA (uPA(-/-)) or uPAR (uPAR(-/-)) abrogates functional recovery after AIS. Treatment with recombinant uPA after ischemic stroke induces neurological recovery in wild-type and uPA(-/-) but not in uPAR(-/-) mice. Diffusion tensor imaging studies indicate that uPA(-/-) mice have increased water diffusivity and decreased anisotropy associated with impaired dendritic spine recovery and decreased length of distal neurites in the peri-infarct cortex. We found that the excitotoxic injury induces the clustering of uPAR in dendritic varicosities, and that the binding of uPA to uPAR promotes the reorganization of the actin cytoskeleton and re-emergence of dendritic filopodia from uPAR-enriched varicosities. This effect is independent of uPA's proteolytic properties and instead is mediated by Rac-regulated profilin expression and cofilin phosphorylation. Our data indicate that binding of uPA to uPAR promotes dendritic spine recovery and improves functional outcome following AIS.

Decoding reveals the contents of visual working memory in early visual areas.

Visual working memory provides an essential link between perception and higher cognitive functions, allowing for the active maintenance of information about stimuli no longer in view. Research suggests that sustained activity in higher-order prefrontal, parietal, inferotemporal and lateral occipital areas supports visual maintenance, and may account for the limited capacity of working memory to hold up to 3-4 items. Because higher-order areas lack the visual selectivity of early sensory areas, it has remained unclear how observers can remember specific visual features, such as the precise orientation of a grating, with minimal decay in performance over delays of many seconds. One proposal is that sensory areas serve to maintain fine-tuned feature information, but early visual areas show little to no sustained activity over prolonged delays. Here we show that orientations held in working memory can be decoded from activity patterns in the human visual cortex, even when overall levels of activity are low. Using functional magnetic resonance imaging and pattern classification methods, we found that activity patterns in visual areas V1-V4 could predict which of two oriented gratings was held in memory with mean accuracy levels upwards of 80%, even in participants whose activity fell to baseline levels after a prolonged delay. These orientation-selective activity patterns were sustained throughout the delay period, evident in individual visual areas, and similar to the responses evoked by unattended, task-irrelevant gratings. Our results demonstrate that early visual areas can retain specific information about visual features held in working memory, over periods of many seconds when no physical stimulus is present.