Distinct feedforward and feedback pathways for cell-type specific attention effects.

Selective attention is thought to depend on enhanced firing activity in extrastriate areas. Theories suggest that this enhancement depends on selective inter-areal communication via gamma (30-80 Hz) phase-locking. To test this, we simultaneously recorded from different cell types and cortical layers of macaque V1 and V4. We find that while V1-V4 gamma phase-locking between local field potentials increases with attention, the V1 gamma rhythm does not engage V4 excitatory-neurons, but only fast-spiking interneurons in L4 of V4. By contrast, attention enhances V4 spike-rates in both excitatory and inhibitory cells, most strongly in L2/3. The rate increase in L2/3 of V4 precedes V1 in time. These findings suggest enhanced signal transmission with attention does not depend on inter-areal gamma phase-locking and show that the endogenous gamma rhythm has cell-type- and layer-specific effects on downstream target areas. Similar findings were made in the mouse visual system, based on opto-tagging of identified interneurons.

Distributed representations of prediction error signals across the cortical hierarchy are synergistic.

A relevant question concerning inter-areal communication in the cortex is whether these interactions are synergistic. Synergy refers to the complementary effect of multiple brain signals conveying more information than the sum of each isolated signal. Redundancy, on the other hand, refers to the common information shared between brain signals. Here, we dissociated cortical interactions encoding complementary information (synergy) from those sharing common information (redundancy) during prediction error (PE) processing. We analyzed auditory and frontal electrocorticography (ECoG) signals in five common awake marmosets performing two distinct auditory oddball tasks and investigated to what extent event-related potentials (ERP) and broadband (BB) dynamics encoded synergistic and redundant information about PE processing. The information conveyed by ERPs and BB signals was synergistic even at lower stages of the hierarchy in the auditory cortex and between auditory and frontal regions. Using a brain-constrained neural network, we simulated the synergy and redundancy observed in the experimental results and demonstrated that the emergence of synergy between auditory and frontal regions requires the presence of strong, long-distance, feedback, and feedforward connections. These results indicate that distributed representations of PE signals across the cortical hierarchy can be highly synergistic.

A novel feature-scrambling approach reveals the capacity of convolutional neural networks to learn spatial relations.

Convolutional neural networks (CNNs) are one of the most successful computer vision systems to solve object recognition. Furthermore, CNNs have major applications in understanding the nature of visual representations in the human brain. Yet it remains poorly understood how CNNs actually make their decisions, what the nature of their internal representations is, and how their recognition strategies differ from humans. Specifically, there is a major debate about the question of whether CNNs primarily rely on surface regularities of objects, or whether they are capable of exploiting the spatial arrangement of features, similar to humans. Here, we develop a novel feature-scrambling approach to explicitly test whether CNNs use the spatial arrangement of features (i.e. object parts) to classify objects. We combine this approach with a systematic manipulation of effective receptive field sizes of CNNs as well as minimal recognizable configurations (MIRCs) analysis. In contrast to much previous literature, we provide evidence that CNNs are in fact capable of using relatively long-range spatial relationships for object classification. Moreover, the extent to which CNNs use spatial relationships depends heavily on the dataset, e.g. texture vs. sketch. In fact, CNNs even use different strategies for different classes within heterogeneous datasets (ImageNet), suggesting CNNs have a continuous spectrum of classification strategies. Finally, we show that CNNs learn the spatial arrangement of features only up to an intermediate level of granularity, which suggests that intermediate rather than global shape features provide the optimal trade-off between sensitivity and specificity in object classification. These results provide novel insights into the nature of CNN representations and the extent to which they rely on the spatial arrangement of features for object classification.

Sequence anticipation and spike-timing-dependent plasticity emerge from a predictive learning rule.

Intelligent behavior depends on the brain's ability to anticipate future events. However, the learning rules that enable neurons to predict and fire ahead of sensory inputs remain largely unknown. We propose a plasticity rule based on predictive processing, where the neuron learns a low-rank model of the synaptic input dynamics in its membrane potential. Neurons thereby amplify those synapses that maximally predict other synaptic inputs based on their temporal relations, which provide a solution to an optimization problem that can be implemented at the single-neuron level using only local information. Consequently, neurons learn sequences over long timescales and shift their spikes towards the first inputs in a sequence. We show that this mechanism can explain the development of anticipatory signalling and recall in a recurrent network. Furthermore, we demonstrate that the learning rule gives rise to several experimentally observed STDP (spike-timing-dependent plasticity) mechanisms. These findings suggest prediction as a guiding principle to orchestrate learning and synaptic plasticity in single neurons.

Self-Relevance Predicts the Aesthetic Appeal of Real and Synthetic Artworks Generated via Neural Style Transfer.

What determines the aesthetic appeal of artworks? Recent work suggests that aesthetic appeal can, to some extent, be predicted from a visual artwork's image features. Yet a large fraction of variance in aesthetic ratings remains unexplained and may relate to individual preferences. We hypothesized that an artwork's aesthetic appeal depends strongly on self-relevance. In a first study (N = 33 adults, online replication N = 208), rated aesthetic appeal for real artworks was positively predicted by rated self-relevance. In a second experiment (N = 45 online), we created synthetic, self-relevant artworks using deep neural networks that transferred the style of existing artworks to photographs. Style transfer was applied to self-relevant photographs selected to reflect participant-specific attributes such as autobiographical memories. Self-relevant, synthetic artworks were rated as more aesthetically appealing than matched control images, at a level similar to human-made artworks. Thus, self-relevance is a key determinant of aesthetic appeal, independent of artistic skill and image features.

SpikeShip: A method for fast, unsupervised discovery of high-dimensional neural spiking patterns.

Neural coding and memory formation depend on temporal spiking sequences that span high-dimensional neural ensembles. The unsupervised discovery and characterization of these spiking sequences requires a suitable dissimilarity measure to spiking patterns, which can then be used for clustering and decoding. Here, we present a new dissimilarity measure based on optimal transport theory called SpikeShip, which compares multi-neuron spiking patterns based on all the relative spike-timing relationships among neurons. SpikeShip computes the optimal transport cost to make all the relative spike-timing relationships (across neurons) identical between two spiking patterns. We show that this transport cost can be decomposed into a temporal rigid translation term, which captures global latency shifts, and a vector of neuron-specific transport flows, which reflect inter-neuronal spike timing differences. SpikeShip can be effectively computed for high-dimensional neuronal ensembles, has a low (linear) computational cost that has the same order as the spike count, and is sensitive to higher-order correlations. Furthermore, SpikeShip is binless, can handle any form of spike time distributions, is not affected by firing rate fluctuations, can detect patterns with a low signal-to-noise ratio, and can be effectively combined with a sliding window approach. We compare the advantages and differences between SpikeShip and other measures like SPIKE and Victor-Purpura distance. We applied SpikeShip to large-scale Neuropixel recordings during spontaneous activity and visual encoding. We show that high-dimensional spiking sequences detected via SpikeShip reliably distinguish between different natural images and different behavioral states. These spiking sequences carried complementary information to conventional firing rate codes. SpikeShip opens new avenues for studying neural coding and memory consolidation by rapid and unsupervised detection of temporal spiking patterns in high-dimensional neural ensembles.

Attentional modulation of inter-areal coherence explained by frequency shifts.

Inter-areal coherence has been hypothesized as a mechanism for inter-areal communication. Indeed, empirical studies have observed an increase in inter-areal coherence with attention. Yet, the mechanisms underlying changes in coherence remain largely unknown. Both attention and stimulus salience are associated with shifts in the peak frequency of gamma oscillations in V1, which suggests that the frequency of oscillations may play a role in facilitating changes in inter-areal communication and coherence. In this study, we used computational modeling to investigate how the peak frequency of a sender influences inter-areal coherence. We show that changes in the magnitude of coherence are largely determined by the peak frequency of the sender. However, the pattern of coherence depends on the intrinsic properties of the receiver, specifically whether the receiver integrates or resonates with its synaptic inputs. Because resonant receivers are frequency-selective, resonance has been proposed as a mechanism for selective communication. However, the pattern of coherence changes produced by a resonant receiver is inconsistent with empirical studies. By contrast, an integrator receiver does produce the pattern of coherence with frequency shifts in the sender observed in empirical studies. These results indicate that coherence can be a misleading measure of inter-areal interactions. This led us to develop a new measure of inter-areal interactions, which we refer to as Explained Power. We show that Explained Power maps directly to the signal transmitted by the sender filtered by the receiver, and thus provides a method to quantify the true signals transmitted between the sender and receiver. Together, these findings provide a model of changes in inter-areal coherence and Granger-causality as a result of frequency shifts.

Cell-type-specific propagation of visual flicker.

Rhythmic flicker stimulation has gained interest as a treatment for neurodegenerative diseases and as a method for frequency tagging neural activity. Yet, little is known about the way in which flicker-induced synchronization propagates across cortical levels and impacts different cell types. Here, we use Neuropixels to record from the lateral geniculate nucleus (LGN), the primary visual cortex (V1), and CA1 in mice while presenting visual flicker stimuli. LGN neurons show strong phase locking up to 40 Hz, whereas phase locking is substantially weaker in V1 and is absent in CA1. Laminar analyses reveal an attenuation of phase locking at 40 Hz for each processing stage. Gamma-rhythmic flicker predominantly entrains fast-spiking interneurons. Optotagging experiments show that these neurons correspond to either parvalbumin (PV+) or narrow-waveform somatostatin (Sst+) neurons. A computational model can explain the observed differences based on the neurons' capacitative low-pass filtering properties. In summary, the propagation of synchronized activity and its effect on distinct cell types strongly depend on its frequency.

Spike-based coupling between single neurons and populations across rat sensory cortices, perirhinal cortex, and hippocampus.

Cortical computations require coordination of neuronal activity within and across multiple areas. We characterized spiking relationships within and between areas by quantifying coupling of single neurons to population firing patterns. Single-neuron population coupling (SNPC) was investigated using ensemble recordings from hippocampal CA1 region and somatosensory, visual, and perirhinal cortices. Within-area coupling was heterogeneous across structures, with area CA1 showing higher levels than neocortical regions. In contrast to known anatomical connectivity, between-area coupling showed strong firing coherence of sensory neocortices with CA1, but less with perirhinal cortex. Cells in sensory neocortices and CA1 showed positive correlations between within- and between-area coupling; these were weaker for perirhinal cortex. All four areas harbored broadcasting cells, connecting to multiple external areas, which was uncorrelated to within-area coupling strength. When examining correlations between SNPC and spatial coding, we found that, if such correlations were significant, they were negative. This result was consistent with an overall preservation of SNPC across different brain states, suggesting a strong dependence on intrinsic network connectivity. Overall, SNPC offers an important window on cell-to-population synchronization in multi-area networks. Instead of pointing to specific information-coding functions, our results indicate a primary function of SNPC in dynamically organizing communication in systems composed of multiple, interconnected areas.

Principles of large-scale neural interactions.

What mechanisms underlie flexible inter-areal communication in the cortex? We consider four mechanisms for temporal coordination and their contributions to communication: (1) Oscillatory synchronization (communication-through-coherence); (2) communication-through-resonance; (3) non-linear integration; and (4) linear signal transmission (coherence-through-communication). We discuss major challenges for communication-through-coherence based on layer- and cell-type-specific analyses of spike phase-locking, heterogeneity of dynamics across networks and states, and computational models for selective communication. We argue that resonance and non-linear integration are viable alternative mechanisms that facilitate computation and selective communication in recurrent networks. Finally, we consider communication in relation to cortical hierarchy and critically examine the hypothesis that feedforward and feedback communication use fast (gamma) and slow (alpha/beta) frequencies, respectively. Instead, we propose that feedforward propagation of prediction errors relies on the non-linear amplification of aperiodic transients, whereas gamma and beta rhythms represent rhythmic equilibrium states that facilitate sustained and efficient information encoding and amplification of short-range feedback via resonance.

Developmental loss of ErbB4 in PV interneurons disrupts state-dependent cortical circuit dynamics.

GABAergic inhibition plays an important role in the establishment and maintenance of cortical circuits during development. Neuregulin 1 (Nrg1) and its interneuron-specific receptor ErbB4 are key elements of a signaling pathway critical for the maturation and proper synaptic connectivity of interneurons. Using conditional deletions of the ERBB4 gene in mice, we tested the role of this signaling pathway at two developmental timepoints in parvalbumin-expressing (PV) interneurons, the largest subpopulation of cortical GABAergic cells. Loss of ErbB4 in PV interneurons during embryonic, but not late postnatal development leads to alterations in the activity of excitatory and inhibitory cortical neurons, along with severe disruption of cortical temporal organization. These impairments emerge by the end of the second postnatal week, prior to the complete maturation of the PV interneurons themselves. Early loss of ErbB4 in PV interneurons also results in profound dysregulation of excitatory pyramidal neuron dendritic architecture and a redistribution of spine density at the apical dendritic tuft. In association with these deficits, excitatory cortical neurons exhibit normal tuning for sensory inputs, but a loss of state-dependent modulation of the gain of sensory responses. Together these data support a key role for early developmental Nrg1/ErbB4 signaling in PV interneurons as a powerful mechanism underlying the maturation of both the inhibitory and excitatory components of cortical circuits.

Coherence fails to reliably capture inter-areal interactions in bidirectional neural systems with transmission delays.

Accurately measuring and quantifying the underlying interactions between brain areas is crucial for understanding the flow of information in the brain. Of particular interest in the field of electrophysiology is the analysis and characterization of the spectral properties of these interactions. Coherence and Granger-Geweke causality are well-established, commonly used methods for quantifying inter-areal interactions, and are thought to reflect the strength of inter-areal interactions. Here we show that the application of both methods to bidirectional systems with transmission delays is problematic, especially for coherence. Under certain circumstances, coherence can be completely abolished despite there being a true underlying interaction. This problem occurs due to interference caused in the computation of coherence, and is an artifact of the method. We motivate an understanding of the problem through computational modelling and numerical simulations. In addition, we have developed two methods that can recover the true bidirectional interactions in the presence of transmission delays.

Improved visualization of high-dimensional data using the distance-of-distance transformation.

Dimensionality reduction tools like t-SNE and UMAP are widely used for high-dimensional data analysis. For instance, these tools are applied in biology to describe spiking patterns of neuronal populations or the genetic profiles of different cell types. Here, we show that when data include noise points that are randomly scattered within a high-dimensional space, a "scattering noise problem" occurs in the low-dimensional embedding where noise points overlap with the cluster points. We show that a simple transformation of the original distance matrix by computing a distance between neighbor distances alleviates this problem and identifies the noise points as a separate cluster. We apply this technique to high-dimensional neuronal spike sequences, as well as the representations of natural images by convolutional neural network units, and find an improvement in the constructed low-dimensional embedding. Thus, we present an improved dimensionality reduction technique for high-dimensional data containing noise points.

Spontaneous variability in gamma dynamics described by a damped harmonic oscillator driven by noise.

Circuits of excitatory and inhibitory neurons generate gamma-rhythmic activity (30-80 Hz). Gamma-cycles show spontaneous variability in amplitude and duration. To investigate the mechanisms underlying this variability, we recorded local-field-potentials (LFPs) and spikes from awake macaque V1. We developed a noise-robust method to detect gamma-cycle amplitudes and durations, which showed a weak but positive correlation. This correlation, and the joint amplitude-duration distribution, is well reproduced by a noise-driven damped harmonic oscillator. This model accurately fits LFP power-spectra, is equivalent to a linear, noise-driven E-I circuit, and recapitulates two additional features of gamma: (1) Amplitude-duration correlations decrease with oscillation strength; (2) amplitudes and durations exhibit strong and weak autocorrelations, respectively, depending on oscillation strength. Finally, longer gamma-cycles are associated with stronger spike-synchrony, but lower spike-rates in both (putative) excitatory and inhibitory neurons. In sum, V1 gamma-dynamics are well described by the simplest possible model of gamma: A damped harmonic oscillator driven by noise.

Deficient Recurrent Cortical Processing in Congenital Deafness.

The influence of sensory experience on cortical feedforward and feedback interactions has rarely been studied in the auditory cortex. Previous work has documented a dystrophic effect of deafness in deep cortical layers, and a reduction of interareal couplings between primary and secondary auditory areas in congenital deafness which was particularly pronounced in the top-down direction (from the secondary to the primary area). In the present study, we directly quantified the functional interaction between superficial (supragranular, I to III) and deep (infragranular, V and VI) layers of feline's primary auditory cortex A1, and also between superficial/deep layers of A1 and a secondary auditory cortex, namely the posterior auditory field (PAF). We compared adult hearing cats under acoustic stimulation and cochlear implant (CI) stimulation to adult congenitally deaf cats (CDC) under CI stimulation. Neuronal activity was recorded from auditory fields A1 and PAF simultaneously with two NeuroNexus electrode arrays. We quantified the spike field coherence (i.e., the statistical dependence of spike trains at one electrode with local field potentials on another electrode) using pairwise phase consistency (PPC). Both the magnitude as well as the preferred phase of synchronization was analyzed. The magnitude of PPC was significantly smaller in CDCs than in controls. Furthermore, controls showed no significant difference between the preferred phase of synchronization between supragranular and infragranular layers, both in acoustic and electric stimulation. In CDCs, however, there was a large difference in the preferred phase between supragranular and infragranular layers. These results demonstrate a loss of synchrony and for the first time directly document a functional decoupling of the interaction between supragranular and infragranular layers of the primary auditory cortex in congenital deafness. Since these are key for the influence of top-down to bottom-up computations, the results suggest a loss of recurrent cortical processing in congenital deafness and explain the outcomes of previous studies by deficits in intracolumnar microcircuitry.

Predictive coding of natural images by V1 firing rates and rhythmic synchronization.

Predictive coding is an important candidate theory of self-supervised learning in the brain. Its central idea is that sensory responses result from comparisons between bottom-up inputs and contextual predictions, a process in which rates and synchronization may play distinct roles. We recorded from awake macaque V1 and developed a technique to quantify stimulus predictability for natural images based on self-supervised, generative neural networks. We find that neuronal firing rates were mainly modulated by the contextual predictability of higher-order image features, which correlated strongly with human perceptual similarity judgments. By contrast, V1 gamma (γ)-synchronization increased monotonically with the contextual predictability of low-level image features and emerged exclusively for larger stimuli. Consequently, γ-synchronization was induced by natural images that are highly compressible and low-dimensional. Natural stimuli with low predictability induced prominent, late-onset beta (ß)-synchronization, likely reflecting cortical feedback. Our findings reveal distinct roles of synchronization and firing rates in the predictive coding of natural images.

Stimulus-specific plasticity of macaque V1 spike rates and gamma.

When a visual stimulus is repeated, average neuronal responses typically decrease, yet they might maintain or even increase their impact through increased synchronization. Previous work has found that many repetitions of a grating lead to increasing gamma-band synchronization. Here, we show in awake macaque area V1 that both repetition-related reductions in firing rate and increases in gamma are specific to the repeated stimulus. These effects show some persistence on the timescale of minutes. Gamma increases are specific to the presented stimulus location. Further, repetition effects on gamma and on firing rates generalize to images of natural objects. These findings support the notion that gamma-band synchronization subserves the adaptive processing of repeated stimulus encounters.

A mechanism for inter-areal coherence through communication based on connectivity and oscillatory power.

Inter-areal coherence between field potentials is a widespread phenomenon in cortex. Coherence has been hypothesized to reflect phase-synchronization between oscillators and flexibly gate communication according to behavioral and cognitive demands. We reveal an alternative mechanism where coherence is not the cause but the consequence of communication and naturally emerges because spiking activity in a sending area causes post-synaptic potentials both in the same and in other areas. Consequently, coherence depends in a lawful manner on power and phase-locking in the sender and connectivity. Changes in oscillatory power explained prominent changes in fronto-parietal and LGN-V1 coherence across behavioral conditions. Optogenetic experiments and excitatory-inhibitory network simulations identified afferent synaptic inputs rather than spiking entrainment as the principal determinant of coherence. These findings suggest that unique spectral profiles of different brain areas automatically give rise to large-scale coherence patterns that follow anatomical connectivity and continuously reconfigure as a function of behavior and cognition.

Brain rhythms define distinct interaction networks with differential dependence on anatomy.

Cognitive functions are subserved by rhythmic neuronal synchronization across widely distributed brain areas. In 105 area pairs, we investigated functional connectivity (FC) through coherence, power correlation, and Granger causality (GC) in the theta, beta, high-beta, and gamma rhythms. Between rhythms, spatial FC patterns were largely independent. Thus, the rhythms defined distinct interaction networks. Importantly, networks of coherence and GC were not explained by the spatial distributions of the strengths of the rhythms. Those networks, particularly the GC networks, contained clear modules, with typically one dominant rhythm per module. To understand how this distinctiveness and modularity arises on a common anatomical backbone, we correlated, across 91 area pairs, the metrics of functional interaction with those of anatomical projection strength. Anatomy was primarily related to coherence and GC, with the largest effect sizes for GC. The correlation differed markedly between rhythms, being less pronounced for the beta and strongest for the gamma rhythm.