Visual perception of highly memorable images is mediated by a distributed network of ventral visual regions that enable a late memorability response.

Behavioral and neuroscience studies in humans and primates have shown that memorability is an intrinsic property of an image that predicts its strength of encoding into and retrieval from memory. While previous work has independently probed when or where this memorability effect may occur in the human brain, a description of its spatiotemporal dynamics is missing. Here, we used representational similarity analysis (RSA) to combine functional magnetic resonance imaging (fMRI) with source-estimated magnetoencephalography (MEG) to simultaneously measure when and where the human cortex is sensitive to differences in image memorability. Results reveal that visual perception of High Memorable images, compared to Low Memorable images, recruits a set of regions of interest (ROIs) distributed throughout the ventral visual cortex: a late memorability response (from around 300 ms) in early visual cortex (EVC), inferior temporal cortex, lateral occipital cortex, fusiform gyrus, and banks of the superior temporal sulcus. Image memorability magnitude results are represented after high-level feature processing in visual regions and reflected in classical memory regions in the medial temporal lobe (MTL). Our results present, to our knowledge, the first unified spatiotemporal account of visual memorability effect across the human cortex, further supporting the levels-of-processing theory of perception and memory.

Emergence of Visual Center-Periphery Spatial Organization in Deep Convolutional Neural Networks.

Research at the intersection of computer vision and neuroscience has revealed hierarchical correspondence between layers of deep convolutional neural networks (DCNNs) and cascade of regions along human ventral visual cortex. Recently, studies have uncovered emergence of human interpretable concepts within DCNNs layers trained to identify visual objects and scenes. Here, we asked whether an artificial neural network (with convolutional structure) trained for visual categorization would demonstrate spatial correspondences with human brain regions showing central/peripheral biases. Using representational similarity analysis, we compared activations of convolutional layers of a DCNN trained for object and scene categorization with neural representations in human brain visual regions. Results reveal a brain-like topographical organization in the layers of the DCNN, such that activations of layer-units with central-bias were associated with brain regions with foveal tendencies (e.g. fusiform gyrus), and activations of layer-units with selectivity for image backgrounds were associated with cortical regions showing peripheral preference (e.g. parahippocampal cortex). The emergence of a categorical topographical correspondence between DCNNs and brain regions suggests these models are a good approximation of the perceptual representation generated by biological neural networks.

Reliability and Generalizability of Similarity-Based Fusion of MEG and fMRI Data in Human Ventral and Dorsal Visual Streams.

To build a representation of what we see, the human brain recruits regions throughout the visual cortex in cascading sequence. Recently, an approach was proposed to evaluate the dynamics of visual perception in high spatiotemporal resolution at the scale of the whole brain. This method combined functional magnetic resonance imaging (fMRI) data with magnetoencephalography (MEG) data using representational similarity analysis and revealed a hierarchical progression from primary visual cortex through the dorsal and ventral streams. To assess the replicability of this method, we here present the results of a visual recognition neuro-imaging fusion experiment and compare them within and across experimental settings. We evaluated the reliability of this method by assessing the consistency of the results under similar test conditions, showing high agreement within participants. We then generalized these results to a separate group of individuals and visual input by comparing them to the fMRI-MEG fusion data of Cichy et al (2016), revealing a highly similar temporal progression recruiting both the dorsal and ventral streams. Together these results are a testament to the reproducibility of the fMRI-MEG fusion approach and allows for the interpretation of these spatiotemporal dynamic in a broader context.

The perceptual neural trace of memorable unseen scenes.

Some scenes are more memorable than others: they cement in minds with consistencies across observers and time scales. While memory mechanisms are traditionally associated with the end stages of perception, recent behavioral studies suggest that the features driving these memorability effects are extracted early on, and in an automatic fashion. This raises the question: is the neural signal of memorability detectable during early perceptual encoding phases of visual processing? Using the high temporal resolution of magnetoencephalography (MEG), during a rapid serial visual presentation (RSVP) task, we traced the neural temporal signature of memorability across the brain. We found an early and prolonged memorability related signal under a challenging ultra-rapid viewing condition, across a network of regions in both dorsal and ventral streams. This enhanced encoding could be the key to successful storage and recognition.

Gist Perception of Image Composition in Abstract Artworks.

Most recent studies in experimental aesthetics have focused on the cognitive processing of visual artworks. In contrast, the perception of formal compositional features of artworks has been studied less extensively. Here, we investigated whether fast and automatic processing of artistic image composition can lead to a stable and consistent aesthetic evaluation when cognitive processing is minimized or absent. To this aim, we compared aesthetic ratings on abstract artworks and their shuffled counterparts in a gist experiment. Results show that exposure times as short as 50 ms suffice for the participants to reach a stable and consistent rating on how ordered and harmonious the abstract stimuli were. Moreover, the rating scores for the 50 ms exposure time exhibited similar dependencies on image type and self-similarity and a similar pattern of correlations between different rating terms, as the rating scores for the long exposure time (3,000 ms). Ratings were less consistent for the term interesting and inconsistent for the term pleasing. Our results are compatible with a model of aesthetic experience, in which the early perceptual processing of the formal aspects of visual artworks can lead to a consistent aesthetic judgment, even if there is no cognitive contribution to this judgment.

In the Eye of the Beholder: Rapid Visual Perception of Real-Life Scenes by Young Adults with and Without ASD.

Typically developing (TD) adults are able to extract global information from natural images and to categorize them within a single glance. This study aimed at extending these findings to individuals with autism spectrum disorder (ASD) using a free description open-encoding paradigm. Participants were asked to freely describe what they saw when looking at briefly presented real-life photographs. Our results show subtle but consistent group-level differences. More specifically, individuals with ASD spontaneously reported the presence of people in the display less frequently than TD participants, and they grasped the gist of the scene less well. These findings argue for a less efficient rapid feedforward processing of global semantic aspects and a less spontaneous interpretation of socially salient information in ASD.

Consecutive TMS-fMRI reveals an inverse relationship in BOLD signal between object and scene processing.

The human visual system is capable of recognizing an infinite number of scenes containing an abundance of rich visual information. There are several cortical regions associated with the representation of a scene, including those specialized for object processing (the lateral occipital area [LO]) and for the spatial layout of scenes (the parahippocampal place area). Although behavioral studies have demonstrated that these image categories (scenes and objects) exert an influence on each other such that scene context can facilitate object identification or that scene categorization can be impaired by the presence of a salient object, little is known about the apparent cortical interactions involved in building the conscious representation of a complete scene. It has been shown that transcranial magnetic stimulation (TMS) to the left LO disrupts object categorization but facilitates scene categorization. Here, we show that this effect is also reflected by changes in the BOLD signal such that TMS to the left LO decreases BOLD signal at the stimulation site (LO) while viewing objects and increases BOLD signal in the left PPA when viewing scenes. This suggests that these regions, although likely not on a strict hierarchy of bottom-up coding, share functional communication likely in the form of inhibitory connections.

TMS to the "occipital face area" affects recognition but not categorization of faces.

The human cortical system for face perception is comprised of a network of connected regions including the middle fusiform gyrus ("fusiform face area" or FFA), the inferior occipital cortex ("occipital face area" or OFA), and the superior temporal sulcus. The traditional hierarchical feedforward model of visual processing suggests information flows from early visual cortex to the OFA for initial face feature analysis to higher order regions including the FFA for identity recognition. However, patient data suggest an alternative model. Patients with acquired prosopagnosia, an inability to visually recognize faces, have been documented with lesions to the OFA but who nevertheless show face-selective activation in the FFA. Moreover, their ability to categorize faces remains intact. This suggests that the FFA is not solely responsible for face recognition and the network is not strictly hierarchical, but may be organized in a reverse hierarchical fashion. We used transcranial magnetic stimulation (TMS) to temporarily disrupt processing in the OFA in neurologically-intact individuals and found participants' ability to categorize intact versus scrambled faces was unaffected, however face identity discrimination was significantly impaired. This suggests that face categorization but not recognition can occur without the "earlier" OFA being online and indicates that "lower level" face category processing may be assumed by other intact face network regions such as the FFA. These results are consistent with the patient data and support a non-hierarchical, global-to-local model with re-entrant connections between the OFA and other face processing areas.

Transcranial magnetic stimulation to the transverse occipital sulcus affects scene but not object processing.

Traditionally, it has been theorized that the human visual system identifies and classifies scenes in an object-centered approach, such that scene recognition can only occur once key objects within a scene are identified. Recent research points toward an alternative approach, suggesting that the global image features of a scene are sufficient for the recognition and categorization of a scene. We have previously shown that disrupting object processing with repetitive TMS to object-selective cortex enhances scene processing possibly through a release of inhibitory mechanisms between object and scene pathways [Mullin, C. R., & Steeves, J. K. E. TMS to the lateral occipital cortex disrupts object processing but facilitates scene processing. Journal of Cognitive Neuroscience, 23, 4174-4184, 2011]. Here we show the effects of TMS to the transverse occipital sulcus (TOS), an area implicated in scene perception, on scene and object processing. TMS was delivered to the TOS or the vertex (control site) while participants performed an object and scene natural/nonnatural categorization task. Transiently interrupting the TOS resulted in significantly lower accuracies for scene categorization compared with control conditions. This demonstrates a causal role of the TOS in scene processing and indicates its importance, in addition to the parahippocampal place area and retrosplenial cortex, in the scene processing network. Unlike TMS to object-selective cortex, which facilitates scene categorization, disrupting scene processing through stimulation of the TOS did not affect object categorization. Further analysis revealed a higher proportion of errors for nonnatural scenes that led us to speculate that the TOS may be involved in processing the higher spatial frequency content of a scene. This supports a nonhierarchical model of scene recognition.

TMS to the lateral occipital cortex disrupts object processing but facilitates scene processing.

The study of brain-damaged patients and advancements in neuroimaging have lead to the discovery of discrete brain regions that process visual image categories, such as objects and scenes. However, how these visual image categories interact remains unclear. For example, is scene perception simply an extension of object perception, or can global scene "gist" be processed independently of its component objects? Specifically, when recognizing a scene such as an "office," does one need to first recognize its individual objects, such as the desk, chair, lamp, pens, and paper to build up the representation of an "office" scene? Here, we show that temporary interruption of object processing through repetitive TMS to the left lateral occipital cortex (LO), an area known to selectively process objects, impairs object categorization but surprisingly facilitates scene categorization. This result was replicated in a second experiment, which assessed the temporal dynamics of this disruption and facilitation. We further showed that repetitive TMS to left LO significantly disrupted object processing but facilitated scene processing when stimulation was administered during the first 180 msec of the task. This demonstrates that the visual system retains the ability to process scenes during disruption to object processing. Moreover, the facilitation of scene processing indicates disinhibition of areas involved in global scene processing, likely caused by disrupting inhibitory contributions from the LO. These findings indicate separate but interactive pathways for object and scene processing and further reveal a network of inhibitory connections between these visual brain regions.

Sex differences in face processing are mediated by handedness and sexual orientation.

Previous research has demonstrated sex differences in face processing at both neural and behavioural levels. The present study examined the role of handedness and sexual orientation as mediators of this effect. We compared the performance of LH (left-handed) and RH (right-handed) heterosexual and homosexual male and female participants on a face recognition memory task. Our main findings were that homosexual males have better face recognition memory than both heterosexual males and homosexual women. We also demonstrate better face processing in women than in men. Finally, LH heterosexual participants had better face recognition than LH homosexual participants and also tended to be better than RH heterosexual participants. These findings are consistent with differences in the organisation and laterality of face-processing mechanisms as a function of sex, handedness, and sexual orientation.

Preserved striate cortex is not sufficient to support the McCollough effect: evidence from two patients with cerebral achromatopsia.

The McCollough effect (ME) is a colour aftereffect contingent on pattern orientation. This effect is generally thought to be mediated by primary visual cortex (V1) although this has remained the subject of some debate. To determine whether V1 is in fact sufficient to subserve the ME, we compared McCollough adaptation in controls to adaptation in two patients with damage to ventrotemporal cortex, resulting in achromatopsia, but who have spared V1. Each of these patients has some residual colour abilities of which he is unaware. Participants performed a 2AFC orientation-discrimination task for pairs of oblique and vertical/horizontal gratings both before and after adaptation to red/green oblique induction gratings. Successful ME induction would manifest itself as an improvement in oblique-orientation discrimination owing to the additional colour cue after adaptation. Indeed, in controls oblique grating discrimination improved post-adaptation. Further, a subdivision of our control group demonstrated successful ME induction despite a lack of conscious awareness of the added colour cue, indicating that conscious colour awareness is not required for ME induction. The patients, however, did not show improvement in oblique-orientation discrimination, indicating a lack of ME induction. This suggests that V1 must be connected to higher cortical colour areas to drive ME induction.