Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain.

A challenging goal of neural coding is to characterize the neural representations underlying visual perception. To this end, multi-unit activity (MUA) of macaque visual cortex was recorded in a passive fixation task upon presentation of faces and natural images. We analyzed the relationship between MUA and latent representations of state-of-the-art deep generative models, including the conventional and feature-disentangled representations of generative adversarial networks (GANs) (i.e., z- and w-latents of StyleGAN, respectively) and language-contrastive representations of latent diffusion networks (i.e., CLIP-latents of Stable Diffusion). A mass univariate neural encoding analysis of the latent representations showed that feature-disentangled w representations outperform both z and CLIP representations in explaining neural responses. Further, w-latent features were found to be positioned at the higher end of the complexity gradient which indicates that they capture visual information relevant to high-level neural activity. Subsequently, a multivariate neural decoding analysis of the feature-disentangled representations resulted in state-of-the-art spatiotemporal reconstructions of visual perception. Taken together, our results not only highlight the important role of feature-disentanglement in shaping high-level neural representations underlying visual perception but also serve as an important benchmark for the future of neural coding.

Experience shapes chandelier cell function and structure in the visual cortex.

Detailed characterization of interneuron types in primary visual cortex (V1) has greatly contributed to understanding visual perception, yet the role of chandelier cells (ChCs) in visual processing remains poorly characterized. Using viral tracing we found that V1 ChCs predominantly receive monosynaptic input from local layer 5 pyramidal cells and higher-order cortical regions. Two-photon calcium imaging and convolutional neural network modeling revealed that ChCs are visually responsive but weakly selective for stimulus content. In mice running in a virtual tunnel, ChCs respond strongly to events known to elicit arousal, including locomotion and visuomotor mismatch. Repeated exposure of the mice to the virtual tunnel was accompanied by reduced visual responses of ChCs and structural plasticity of ChC boutons and axon initial segment length. Finally, ChCs only weakly inhibited pyramidal cells. These findings suggest that ChCs provide an arousal-related signal to layer 2/3 pyramidal cells that may modulate their activity and/or gate plasticity of their axon initial segments during behaviorally relevant events.

The representation of occluded image regions in area V1 of monkeys and humans.

Neuronal activity in the primary visual cortex (V1) is driven by feedforward input from within the neurons' receptive fields (RFs) and modulated by contextual information in regions surrounding the RF. The effect of contextual information on spiking activity occurs rapidly and is therefore challenging to dissociate from feedforward input. To address this challenge, we recorded the spiking activity of V1 neurons in monkeys viewing either natural scenes or scenes where the information in the RF was occluded, effectively removing the feedforward input. We found that V1 neurons responded rapidly and selectively to occluded scenes. V1 responses elicited by occluded stimuli could be used to decode individual scenes and could be predicted from those elicited by non-occluded images, indicating that there is an overlap between visually driven and contextual responses. We used representational similarity analysis to show that the structure of V1 representations of occluded scenes measured with electrophysiology in monkeys correlates strongly with the representations of the same scenes in humans measured with functional magnetic resonance imaging (fMRI). Our results reveal that contextual influences rapidly alter V1 spiking activity in monkeys over distances of several degrees in the visual field, carry information about individual scenes, and resemble those in human V1. VIDEO ABSTRACT.

Information load dynamically modulates functional brain connectivity during narrative listening.

Narratives are paradigmatic examples of natural language, where nouns represent a proxy of information. Functional magnetic resonance imaging (fMRI) studies revealed the recruitment of temporal cortices during noun processing and the existence of a noun-specific network at rest. Yet, it is unclear whether, in narratives, changes in noun density influence the brain functional connectivity, so that the coupling between regions correlates with information load. We acquired fMRI activity in healthy individuals listening to a narrative with noun density changing over time and measured whole-network and node-specific degree and betweenness centrality. Network measures were correlated with information magnitude with a time-varying approach. Noun density correlated positively with the across-regions average number of connections and negatively with the average betweenness centrality, suggesting the pruning of peripheral connections as information decreased. Locally, the degree of the bilateral anterior superior temporal sulcus (aSTS) was positively associated with nouns. Importantly, aSTS connectivity cannot be explained by changes in other parts of speech (e.g., verbs) or syllable density. Our results indicate that the brain recalibrates its global connectivity as a function of the information conveyed by nouns in natural language. Also, using naturalistic stimulation and network metrics, we corroborate the role of aSTS in noun processing.

Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis.

Object recognition relies on different transformations of the retinal input, carried out by the visual system, that range from local contrast to object shape and category. While some of those transformations are thought to occur at specific stages of the visual hierarchy, the features they represent are correlated (e.g., object shape and identity) and selectivity for the same feature overlaps in many brain regions. This may be explained either by collinearity across representations or may instead reflect the coding of multiple dimensions by the same cortical population. Moreover, orthogonal and shared components may differently impact distinctive stages of the visual hierarchy. We recorded functional MRI activity while participants passively attended to object images and employed a statistical approach that partitioned orthogonal and shared object representations to reveal their relative impact on brain processing. Orthogonal shape representations (silhouette, curvature, and medial axis) independently explained distinct and overlapping clusters of selectivity in the occitotemporal and parietal cortex. Moreover, we show that the relevance of shared representations linearly increases moving from posterior to anterior regions. These results indicate that the visual cortex encodes shared relations between different features in a topographic fashion and that object shape is encoded along different dimensions, each representing orthogonal features.NEW & NOTEWORTHY There are several possible ways of characterizing the shape of an object. Which shape description better describes our brain responses while we passively perceive objects? Here, we employed three competing shape models to explain brain representations when viewing real objects. We found that object shape is encoded in a multidimensional fashion and thus defined by the interaction of multiple features.

Emotionotopy in the human right temporo-parietal cortex.

Humans use emotions to decipher complex cascades of internal events. However, which mechanisms link descriptions of affective states to brain activity is unclear, with evidence supporting either local or distributed processing. A biologically favorable alternative is provided by the notion of gradient, which postulates the isomorphism between functional representations of stimulus features and cortical distance. Here, we use fMRI activity evoked by an emotionally charged movie and continuous ratings of the perceived emotion intensity to reveal the topographic organization of affective states. Results show that three orthogonal and spatially overlapping gradients encode the polarity, complexity and intensity of emotional experiences in right temporo-parietal territories. The spatial arrangement of these gradients allows the brain to map a variety of affective states within a single patch of cortex. As this organization resembles how sensory regions represent psychophysical properties (e.g., retinotopy), we propose emotionotopy as a principle of emotion coding.

Common spatiotemporal processing of visual features shapes object representation.

Biological vision relies on representations of the physical world at different levels of complexity. Relevant features span from simple low-level properties, as contrast and spatial frequencies, to object-based attributes, as shape and category. However, how these features are integrated into coherent percepts is still debated. Moreover, these dimensions often share common biases: for instance, stimuli from the same category (e.g., tools) may have similar shapes. Here, using magnetoencephalography, we revealed the temporal dynamics of feature processing in human subjects attending to objects from six semantic categories. By employing Relative Weights Analysis, we mitigated collinearity between model-based descriptions of stimuli and showed that low-level properties (contrast and spatial frequencies), shape (medial-axis) and category are represented within the same spatial locations early in time: 100-150 ms after stimulus onset. This fast and overlapping processing may result from independent parallel computations, with categorical representation emerging later than the onset of low-level feature processing, yet before shape coding. Categorical information is represented both before and after shape, suggesting a role for this feature in the refinement of categorical matching.

Long-lived compositional heterogeneities in magma chambers, and implications for volcanic hazard.

Magmas discharged during individual volcanic eruptions commonly display compositional variations interpreted as new arrivals at shallow depth of more primitive, hotter, volatile-rich magma batches mixing with resident, colder, partially degassed magma. Heterogeneities in eruption products are often interpreted as evidence of short times of order tens of hours from new magma arrival to eruption, raising concerns for emergency planning. We show here, through numerical simulations, that magma convection and mixing in a shallow magma chamber can result in long-lived, dynamically stable configurations with coexistence of magmas from nearly pure to variably mixed end-member compositions. Short mixing time scales may therefore relate to sin-eruptive processes, as heterogeneities found in the eruptive products are not necessarily the fingerprint of new magma arrival shortly preceding or triggering the eruption.

Eight Weddings and Six Funerals: An fMRI Study on Autobiographical Memories.

"Autobiographical memory" (AM) refers to remote memories from one's own life. Previous neuroimaging studies have highlighted that voluntary retrieval processes from AM involve different forms of memory and cognitive functions. Thus, a complex and widespread brain functional network has been found to support AM. The present functional magnetic resonance imaging (fMRI) study used a multivariate approach to determine whether neural activity within the AM circuit would recognize memories of real autobiographical events, and to evaluate individual differences in the recruitment of this network. Fourteen right-handed females took part in the study. During scanning, subjects were presented with sentences representing a detail of a highly emotional real event (positive or negative) and were asked to indicate whether the sentence described something that had or had not really happened to them. Group analysis showed a set of cortical areas able to discriminate the truthfulness of the recalled events: medial prefrontal cortex, posterior cingulate/retrosplenial cortex, precuneus, bilateral angular, superior frontal gyri, and early visual cortical areas. Single-subject results showed that the decoding occurred at different time points. No differences were found between recalling a positive or a negative event. Our results show that the entire AM network is engaged in monitoring the veracity of AMs. This process is not affected by the emotional valence of the experience but rather by individual differences in cognitive strategies used to retrieve AMs.

Foreground-Background Segmentation Revealed during Natural Image Viewing.

One of the major challenges in visual neuroscience is represented by foreground-background segmentation. Data from nonhuman primates show that segmentation leads to two distinct, but associated processes: the enhancement of neural activity during figure processing (i.e., foreground enhancement) and the suppression of background-related activity (i.e., background suppression). To study foreground-background segmentation in ecological conditions, we introduce a novel method based on parametric modulation of low-level image properties followed by application of simple computational image-processing models. By correlating the outcome of this procedure with human fMRI activity, measured during passive viewing of 334 natural images, we produced easily interpretable "correlation images" from visual populations. Results show evidence of foreground enhancement in all tested regions, from V1 to lateral occipital complex (LOC), while background suppression occurs in V4 and LOC only. Correlation images derived from V4 and LOC revealed a preserved spatial resolution of foreground textures, indicating a richer representation of the salient part of natural images, rather than a simplistic model of object shape. Our results indicate that scene segmentation occurs during natural viewing, even when individuals are not required to perform any particular task.

Global time-size distribution of volcanic eruptions on Earth.

Volcanic eruptions differ enormously in their size and impacts, ranging from quiet lava flow effusions along the volcano flanks to colossal events with the potential to affect our entire civilization. Knowledge of the time and size distribution of volcanic eruptions is of obvious relevance for understanding the dynamics and behavior of the Earth system, as well as for defining global volcanic risk. From the analysis of recent global databases of volcanic eruptions extending back to more than 2 million years, I show here that the return times of eruptions with similar magnitude follow an exponential distribution. The associated relative frequency of eruptions with different magnitude displays a power law, scale-invariant distribution over at least six orders of magnitude. These results suggest that similar mechanisms subtend to explosive eruptions from small to colossal, raising concerns on the theoretical possibility to predict the magnitude and impact of impending volcanic eruptions.

Modality-independent encoding of individual concepts in the left parietal cortex.

The organization of semantic information in the brain has been mainly explored through category-based models, on the assumption that categories broadly reflect the organization of conceptual knowledge. However, the analysis of concepts as individual entities, rather than as items belonging to distinct superordinate categories, may represent a significant advancement in the comprehension of how conceptual knowledge is encoded in the human brain. Here, we studied the individual representation of thirty concrete nouns from six different categories, across different sensory modalities (i.e., auditory and visual) and groups (i.e., sighted and congenitally blind individuals) in a core hub of the semantic network, the left angular gyrus, and in its neighboring regions within the lateral parietal cortex. Four models based on either perceptual or semantic features at different levels of complexity (i.e., low- or high-level) were used to predict fMRI brain activity using representational similarity encoding analysis. When controlling for the superordinate component, high-level models based on semantic and shape information led to significant encoding accuracies in the intraparietal sulcus only. This region is involved in feature binding and combination of concepts across multiple sensory modalities, suggesting its role in high-level representation of conceptual knowledge. Moreover, when the information regarding superordinate categories is retained, a large extent of parietal cortex is engaged. This result indicates the need to control for the coarse-level categorial organization when performing studies on higher-level processes related to the retrieval of semantic information.

When Neuroscience 'Touches' Architecture: From Hapticity to a Supramodal Functioning of the Human Brain.

In the last decades, the rapid growth of functional brain imaging methodologies allowed cognitive neuroscience to address open questions in philosophy and social sciences. At the same time, novel insights from cognitive neuroscience research have begun to influence various disciplines, leading to a turn to cognition and emotion in the fields of planning and architectural design. Since 2003, the Academy of Neuroscience for Architecture has been supporting 'neuro-architecture' as a way to connect neuroscience and the study of behavioral responses to the built environment. Among the many topics related to multisensory perceptual integration and embodiment, the concept of hapticity was recently introduced, suggesting a pivotal role of tactile perception and haptic imagery in architectural appraisal. Arguments have thus risen in favor of the existence of shared cognitive foundations between hapticity and the supramodal functional architecture of the human brain. Precisely, supramodality refers to the functional feature of defined brain regions to process and represent specific information content in a more abstract way, independently of the sensory modality conveying such information to the brain. Here, we highlight some commonalities and differences between the concepts of hapticity and supramodality according to the distinctive perspectives of architecture and cognitive neuroscience. This comparison and connection between these two different approaches may lead to novel observations in regard to people-environment relationships, and even provide empirical foundations for a renewed evidence-based design theory.

Strain-induced magma fragmentation in explosive eruptions.

Explosive eruptions are the most powerful and destructive type of volcanic activity. These eruptions are characterized by magma fragmentation, the process through which a bubbly or foamy magma is transformed into a gas-pyroclast dispersion. Although magma fragmentation has been investigated both experimentally and theoretically, and the basic transport phenomena that occur in a volcanic conduit have been modelled, the underlying mechanism responsible for magma fragmentation is still poorly understood. This lack of understanding seriously limits our ability to forecast volcanic hazards, preventing reliable discrimination between conditions that lead to explosive and effusive eruptions. Here I develop a model in which a fragmentation criterion, based on a rate-limited crossing of the glass transition, , is incorporated into a multiphase fluid-dynamic description of magma ascent. The numerical results of this model demonstrate the feasibility of strain-induced brittle fragmentation of magma in volcanic eruptions, and reconcile experimental with theoretical studies as well as with the observed volcanic products of large explosive eruptions.