Primate brain: A unique connection between dorsal and ventral visual cortex.

In humans and other primates, vision is subserved by at least two parallel processing streams that are interconnected through a pathway known as the vertical occipital fasciculus. New research reveals that this white matter pathway may be a unique feature of the primate brain.

Decoding dynamic visual scenes across the brain hierarchy.

Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a crucial investigation in neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding-Neuropixels dataset and utilize the capabilities of deep learning neural network models to study neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. Our study reveals that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within the visual cortex and subcortical nuclei, in contrast to a relatively reduced encoding activity within hippocampal neurons. Strikingly, our results unveil a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings corroborate existing knowledge in visual coding related to artificial visual stimuli and illuminate the functional role of these deeper brain regions using dynamic stimuli. Consequently, our results suggest a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding quality of dynamic natural visual scenes represented by neural responses, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Sub-bundle based analysis reveals the role of human optic radiation in visual working memory.

White matter (WM) functional activity has been reliably detected through functional magnetic resonance imaging (fMRI). Previous studies have primarily examined WM bundles as unified entities, thereby obscuring the functional heterogeneity inherent within these bundles. Here, for the first time, we investigate the function of sub-bundles of a prototypical visual WM tract-the optic radiation (OR). We use the 7T retinotopy dataset from the Human Connectome Project (HCP) to reconstruct OR and further subdivide the OR into sub-bundles based on the fiber's termination in the primary visual cortex (V1). The population receptive field (pRF) model is then applied to evaluate the retinotopic properties of these sub-bundles, and the consistency of the pRF properties of sub-bundles with those of V1 subfields is evaluated. Furthermore, we utilize the HCP working memory dataset to evaluate the activations of the foveal and peripheral OR sub-bundles, along with LGN and V1 subfields, during 0-back and 2-back tasks. We then evaluate differences in 2bk-0bk contrast between foveal and peripheral sub-bundles (or subfields), and further examine potential relationships between 2bk-0bk contrast and 2-back task d-prime. The results show that the pRF properties of OR sub-bundles exhibit standard retinotopic properties and are typically similar to the properties of V1 subfields. Notably, activations during the 2-back task consistently surpass those under the 0-back task across foveal and peripheral OR sub-bundles, as well as LGN and V1 subfields. The foveal V1 displays significantly higher 2bk-0bk contrast than peripheral V1. The 2-back task d-prime shows strong correlations with 2bk-0bk contrast for foveal and peripheral OR fibers. These findings demonstrate that the blood oxygen level-dependent (BOLD) signals of OR sub-bundles encode high-fidelity visual information, underscoring the feasibility of assessing WM functional activity at the sub-bundle level. Additionally, the study highlights the role of OR in the top-down processes of visual working memory beyond the bottom-up processes for visual information transmission. Conclusively, this study innovatively proposes a novel paradigm for analyzing WM fiber tracts at the individual sub-bundle level and expands understanding of OR function.

Excitatory Projections of Wide Field Collicular Neurons to the Nucleus of the Optic Tract in the Rat.

The superficial layers of the mammalian superior colliculus (SC) contain neurons that are generally responsive to visual stimuli but can differ considerably in morphology and response properties. To elucidate the structure and function of these neurons, we combined extracellular recording and juxtacellular labeling, detailed anatomical reconstruction, and ultrastructural analysis of the synaptic contacts of labeled neurons, using transmission electron microscopy. Our labeled neurons project to different brainstem nuclei. Of particular importance are neurons that fit the morphological criteria of the wide field (WF) neurons and whose dendrites are horizontally oriented. They display a rather characteristic axonal projection pattern to the nucleus of optic tract (NOT); thus, we call them superior collicular WF projecting to the NOT (SCWFNOT) neurons. We corroborated the morphological characterization of this neuronal type as a distinct neuronal class with the help of unsupervised hierarchical cluster analysis. Our ultrastructural data demonstrate that SCWFNOT neurons establish excitatory connections with their targets in the NOT. Although, in rodents, the literature about the WF neurons has focused on their extensive projection to the lateral posterior nucleus of the thalamus, as a conduit for information to reach the visual association areas of the cortex, our data suggest that this subclass of WF neurons may participate in the optokinetic nystagmus.

Factorized visual representations in the primate visual system and deep neural networks.

Object classification has been proposed as a principal objective of the primate ventral visual stream and has been used as an optimization target for deep neural network models (DNNs) of the visual system. However, visual brain areas represent many different types of information, and optimizing for classification of object identity alone does not constrain how other information may be encoded in visual representations. Information about different scene parameters may be discarded altogether ('invariance'), represented in non-interfering subspaces of population activity ('factorization') or encoded in an entangled fashion. In this work, we provide evidence that factorization is a normative principle of biological visual representations. In the monkey ventral visual hierarchy, we found that factorization of object pose and background information from object identity increased in higher-level regions and strongly contributed to improving object identity decoding performance. We then conducted a large-scale analysis of factorization of individual scene parameters - lighting, background, camera viewpoint, and object pose - in a diverse library of DNN models of the visual system. Models which best matched neural, fMRI, and behavioral data from both monkeys and humans across 12 datasets tended to be those which factorized scene parameters most strongly. Notably, invariance to these parameters was not as consistently associated with matches to neural and behavioral data, suggesting that maintaining non-class information in factorized activity subspaces is often preferred to dropping it altogether. Thus, we propose that factorization of visual scene information is a widely used strategy in brains and DNN models thereof.

When looking at a picture, we can quickly identify a recognizable object, such as an apple, applying a single word label to it. Although extensive neuroscience research has focused on how human and monkey brains achieve this recognition, our understanding of how the brain and brain-like computer models interpret other complex aspects of a visual scene ­ such as object position and environmental context ­ remains incomplete. In particular, it was not clear to what extent object recognition comes at the expense of other important scene details. For example, various aspects of the scene might be processed simultaneously. On the other hand, general object recognition may interfere with processing of such details. To investigate this, Lindsey and Issa analyzed 12 monkey and human brain datasets, as well as numerous computer models, to explore how different aspects of a scene are encoded in neurons and how these aspects are represented by computational models. The analysis revealed that preventing effective separation and retention of information about object pose and environmental context worsened object identification in monkey cortex neurons. In addition, the computer models that were the most brain-like could independently preserve the other scene details without interfering with object identification. The findings suggest that human and monkey high level ventral visual processing systems are capable of representing the environment in a more complex way than previously appreciated. In the future, studying more brain activity data could help to identify how rich the encoded information is and how it might support other functions like spatial navigation. This knowledge could help to build computational models that process the information in the same way, potentially improving their understanding of real-world scenes.

Short-latency preference for faces in primate superior colliculus depends on visual cortex.

Face processing is fundamental to primates and has been extensively studied in higher-order visual cortex. Here, we report that visual neurons in the midbrain superior colliculus (SC) of macaque monkeys display a preference for images of faces. This preference emerges within 40 ms of stimulus onset-well before "face patches" in visual cortex-and, at the population level, can be used to distinguish faces from other visual objects with accuracies of ∼80%. This short-latency face preference in SC depends on signals routed through early visual cortex because inactivating the lateral geniculate nucleus, the key relay from retina to cortex, virtually eliminates visual responses in SC, including face-related activity. These results reveal an unexpected circuit in the primate visual system for rapidly detecting faces in the periphery, complementing the higher-order areas needed for recognizing individual faces.

A prominent vertical occipital white matter fasciculus unique to primate brains.

Vision in humans and other primates enlists parallel processing streams in the dorsal and ventral visual cortex, known to support spatial and object processing, respectively. These streams are bridged, however, by a prominent white matter tract, the vertical occipital fasciculus (VOF), identified in both classical neuroanatomy and recent diffusion-weighted magnetic resonance imaging (dMRI) studies. Understanding the evolution of the VOF may shed light on its origin, function, and role in visually guided behaviors. To this end, we acquired high-resolution dMRI data from the brains of select mammalian species, including anthropoid and strepsirrhine primates, a tree shrew, rodents, and carnivores. In each species, we attempted to delineate the VOF after first locating the optic radiations in the occipital white matter. In all primate species examined, the optic radiation was flanked laterally by a prominent and coherent white matter fasciculus recognizable as the VOF. By contrast, the equivalent analysis applied to four non-primate species from the same superorder as primates (tree shrew, ground squirrel, paca, and rat) failed to reveal white matter tracts in the equivalent location. Clear evidence for a VOF was also absent in two larger carnivore species (ferret and fox). Although we cannot rule out the existence of minor or differently organized homologous fiber pathways in the non-primate species, the results suggest that the VOF has greatly expanded, or possibly emerged, in the primate lineage. This adaptation likely facilitated the evolution of unique visually guided behaviors in primates, with direct impacts on manual object manipulation, social interactions, and arboreal locomotion.

Development of ocular dominance columns across rodents and other species: revisiting the concept of critical period plasticity.

The existence of cortical columns, regarded as computational units underlying both lower and higher-order information processing, has long been associated with highly evolved brains, and previous studies suggested their absence in rodents. However, recent discoveries have unveiled the presence of ocular dominance columns (ODCs) in the primary visual cortex (V1) of Long-Evans rats. These domains exhibit continuity from layer 2 through layer 6, confirming their identity as genuine ODCs. Notably, ODCs are also observed in Brown Norway rats, a strain closely related to wild rats, suggesting the physiological relevance of ODCs in natural survival contexts, although they are lacking in albino rats. This discovery has enabled researchers to explore the development and plasticity of cortical columns using a multidisciplinary approach, leveraging studies involving hundreds of individuals-an endeavor challenging in carnivore and primate species. Notably, developmental trajectories differ depending on the aspect under examination: while the distribution of geniculo-cortical afferent terminals indicates matured ODCs even before eye-opening, consistent with prevailing theories in carnivore/primate studies, examination of cortical neuron spiking activities reveals immature ODCs until postnatal day 35, suggesting delayed maturation of functional synapses which is dependent on visual experience. This developmental gap might be recognized as 'critical period' for ocular dominance plasticity in previous studies. In this article, I summarize cross-species differences in ODCs and geniculo-cortical network, followed by a discussion on the development, plasticity, and evolutionary significance of rat ODCs. I discuss classical and recent studies on critical period plasticity in the venue where critical period plasticity might be a component of experience-dependent development. Consequently, this series of studies prompts a paradigm shift in our understanding of species conservation of cortical columns and the nature of plasticity during the classical critical period.

Hue and orientation pinwheels in macaque area V4.

Area V4 is an intermediate-level area of the macaque visual cortical hierarchy that serves key functions in visual processing by integrating inputs from lower areas such as V1 and V2 and providing feedforward inputs to many higher cortical areas. Previous V4 imaging studies have focused on differential responses to color, orientation, disparity, and motion stimuli, but many details of the spatial organization of significant hue and orientation tuning have not been fully described. We used support vector machine (SVM) decoding of intrinsic cortical single-condition responses to generate high-resolution maps of hue and orientation tuning and to describe the organization of hue and orientation pinwheels in V4. Like V1 and V2, V4 contains maps of orientation that are organized as pinwheels. V4 also contains maps of hue that are organized as pinwheels, whose circular organization more closely represents the perception of hue than is observed in antecedent cortical areas. Unlike V1, where orientation is continuously mapped across the surface, V4 hue and orientation pinwheels are organized in limited numbers of pinwheel sequences. The organization of these sequences and the distance between pinwheels may provide insight into the functional organization of V4. Regions significantly tuned for hue occupy roughly four times that of the orientation, are largely separated from each other, and overlap by roughly 5%. This spatial organization is largely consistent with segregated inputs arising from V2 thin and interstripes. This modular organization of V4 suggests that further integration of color and shape might occur in higher areas in inferotemporal cortical.NEW & NOTEWORTHY The representation of hue and orientation in macaque monkey area V4 was determined by intrinsic cortical imaging of responses to isoluminant hues and achromatic grating stimuli. Vector summation of support vector machine (SVM) decoded single-condition responses was used to generate hue and orientation maps that, like V1 orientation maps, were both characterized by distinct pinwheel patterns. These data suggest that pinwheels are an important structure to represent different stimulus features across multiple visual cortical areas.

Mapping short association fibre connectivity up to V3 in the human brain in vivo.

Short association fibres (SAF) are the most abundant fibre pathways in the human white matter. Until recently, SAF could not be mapped comprehensively in vivo because diffusion weighted magnetic resonance imaging with sufficiently high spatial resolution needed to map these thin and short pathways was not possible. Recent developments in acquisition hardware and sequences allowed us to create a dedicated in vivo method for mapping the SAF based on sub-millimetre spatial resolution diffusion weighted tractography, which we validated in the human primary (V1) and secondary (V2) visual cortex against the expected SAF retinotopic order. Here, we extended our original study to assess the feasibility of the method to map SAF in higher cortical areas by including SAF up to V3. Our results reproduced the expected retinotopic order of SAF in the V2-V3 and V1-V3 stream, demonstrating greater robustness to the shorter V1-V2 and V2-V3 than the longer V1-V3 connections. The demonstrated ability of the method to map higher-order SAF connectivity patterns in vivo is an important step towards its application across the brain.

Microglia are dispensable for experience-dependent refinement of mouse visual circuitry.

To test the hypothesized crucial role of microglia in the developmental refinement of neural circuitry, we depleted microglia from mice of both sexes with PLX5622 and examined the experience-dependent maturation of visual circuitry and function. We assessed retinal function, receptive field tuning of visual cortex neurons, acuity and experience-dependent plasticity. None of these measurements detectibly differed in the absence of microglia, challenging the role of microglia in sculpting neural circuits.

Diversity of visual inputs to Kenyon cells of the Drosophila mushroom body.

The arthropod mushroom body is well-studied as an expansion layer representing olfactory stimuli and linking them to contingent events. However, 8% of mushroom body Kenyon cells in Drosophila melanogaster receive predominantly visual input, and their function remains unclear. Here, we identify inputs to visual Kenyon cells using the FlyWire adult whole-brain connectome. Input repertoires are similar across hemispheres and connectomes with certain inputs highly overrepresented. Many visual neurons presynaptic to Kenyon cells have large receptive fields, while interneuron inputs receive spatially restricted signals that may be tuned to specific visual features. Individual visual Kenyon cells randomly sample sparse inputs from combinations of visual channels, including multiple optic lobe neuropils. These connectivity patterns suggest that visual coding in the mushroom body, like olfactory coding, is sparse, distributed, and combinatorial. However, the specific input repertoire to the smaller population of visual Kenyon cells suggests a constrained encoding of visual stimuli.

Optimization-Based Pairwise Interaction Point Process (O-PIPP): A Precise and Universal Retinal Mosaic Modeling Approach.

Purpose: A retinal mosaic, the spatial organization of a population of homotypic neurons, is thought to sample a specific visual feature into the feedforward visual pathway. The purpose of this study was to propose a universal modeling approach for precisely generating retinal mosaics and overcoming the limitations of previous models, especially in modeling abnormal mosaic patterns under disease conditions. Methods: Here, we developed the optimization-based pairwise interaction point process (O-PIPP). It incorporates optimization techniques into previous simulation approaches, enabling directional control of the simulation process according to the user-designed optimization target. For the convenience of the community, we implemented the O-PIPP approach into a Python package and a website application. Results: We showed that the O-PIPP can generate more precise neural spatial patterns of healthy and diseased mosaics compared to previous phenomenological approaches. Notably, through modeling the retinal neural circuitry with O-PIPP-simulated retinitis pigmentosa cone mosaics, we elucidated how the cone mosaic rearrangement impacted the information processing of ganglion cells. Conclusions: The O-PIPP provides a precise and universal tool to simulate realistic mosaics, which could help to investigate the function of retinal mosaics in vision.

The large-scale organization of shape processing in the ventral and dorsal pathways is dissociable from attention.

The two visual pathways model posits that visual information is processed through two distinct cortical systems: The ventral pathway promotes visual recognition, while the dorsal pathway supports visuomotor control. Recent evidence suggests the dorsal pathway is also involved in shape processing and may contribute to object perception, but it remains unclear whether this sensitivity is independent of attentional mechanisms that were localized to overlapping cortical regions. To address this question, we conducted two fMRI experiments that utilized different parametric scrambling manipulations in which human participants viewed novel objects in different levels of scrambling and were instructed to attend to either the object or to another aspect of the image (e.g. color of the background). Univariate and multivariate analyses revealed that the large-scale organization of shape selectivity along the dorsal and ventral pathways was preserved regardless of the focus of attention. Attention did modulate shape sensitivity, but these effects were similar across the two pathways. These findings support the idea that shape processing is at least partially dissociable from attentional processes and relies on a distributed set of cortical regions across the visual pathways.

Moving object detection based on bioinspired background subtraction.

Flying insects rely mainly upon visual motion to detect and track objects. There has been a lot of research on fly inspired algorithms for object detection, but few have been developed based on visual motion alone. One of the daunting difficulties is that the neural and circuit mechanisms underlying the foreground-background segmentation are still unclear. Our previous modeling study proposed that the lobula held parallel pathways with distinct directional selectivity, each of which could retinotopically discriminate figures moving in its own preferred direction based on relative motion cues. The previous model, however, did not address how the multiple parallel pathways gave the only detection output at their common downstream. Since the preferred directions of the pathways along either horizontal or vertical axis were opposite to each other, the background moving in the opposite direction to an object also activated the corresponding lobula pathway. Indiscriminate or ungated projection from all the pathways to their downstream would mix objects with the moving background, making the previous model fail with non-stationary background. Here, we extend the previous model by proposing that the background motion-dependent gating of individual lobula projections is the key to object detection. Large-field lobula plate tangential cells are hypothesized to perform the gating to realize bioinspired background subtraction. The model is shown to be capable of implementing a robust detection of moving objects in video sequences with either a moving camera that induces translational optic flow or a static camera. The model sheds light on the potential of the concise fly algorithm in real-world applications.

Development of reciprocal connections between the dorsal lateral geniculate nucleus and the thalamic reticular nucleus.

The thalamic reticular nucleus (TRN) serves as an important node between the thalamus and neocortex, regulating thalamocortical rhythms and sensory processing in a state dependent manner. Disruptions in TRN circuitry also figures prominently in several neurodevelopmental disorders including epilepsy, autism, and attentional defects. An understanding of how and when connections between TRN and 1st order thalamic nuclei, such as the dorsal lateral geniculate nucleus (dLGN), develop is lacking. We used the mouse visual thalamus as a model system to study the organization, pattern of innervation and functional responses between TRN and the dLGN. Genetically modified mouse lines were used to visualize and target the feedforward and feedback components of these intra-thalamic circuits and to understand how peripheral input from the retina impacts their development.Retrograde tracing of thalamocortical (TC) afferents through TRN revealed that the modality-specific organization seen in the adult, is present at perinatal ages and seems impervious to the loss of peripheral input. To examine the formation and functional maturation of intrathalamic circuits between the visual sector of TRN and dLGN, we examined when projections from each nuclei arrive, and used an acute thalamic slice preparation along with optogenetic stimulation to assess the maturation of functional synaptic responses. Although thalamocortical projections passed through TRN at birth, feedforward axon collaterals determined by vGluT2 labeling, emerged during the second postnatal week, increasing in density through the third week. Optogenetic stimulation of TC axon collaterals in TRN showed infrequent, weak excitatory responses near the end of week 1. During weeks 2-4, responses became more prevalent, grew larger in amplitude and exhibited synaptic depression during repetitive stimulation. Feedback projections from visual TRN to dLGN began to innervate dLGN as early as postnatal day 2 with weak inhibitory responses emerging during week 1. During week 2-4, inhibitory responses continued to grow larger, showing synaptic depression during repetitive stimulation. During this time TRN inhibition started to suppress TC spiking, having its greatest impact by week 4-6. Using a mutant mouse that lacks retinofugal projections revealed that the absence of retinal input led to an acceleration of TRN innervation of dLGN but had little impact on the development of feedforward projections from dLGN to TRN. Together, these experiments reveal how and when intrathalamic connections emerge during early postnatal ages and provide foundational knowledge to understand the development of thalamocortical network dynamics as well as neurodevelopmental diseases that involve TRN circuitry.

Individual thalamic inhibitory interneurons are functionally specialized toward distinct visual features.

Inhibitory interneurons in the dorsolateral geniculate nucleus (dLGN) are situated at the first central synapse of the image-forming visual pathway, but little is known about their function. Given their anatomy, they are expected to be multiplexors, integrating many different retinal channels along their dendrites. Here, using targeted single-cell-initiated rabies tracing, we found that mouse dLGN interneurons exhibit a degree of retinal input specialization similar to thalamocortical neurons. Some are anatomically highly specialized, for example, toward motion-selective information. Two-photon calcium imaging performed in vivo revealed that interneurons are also functionally specialized. In mice lacking retinal horizontal direction selectivity, horizontal direction selectivity is reduced in interneurons, suggesting a causal link between input and functional specialization. Functional specialization is not only present at interneuron somata but also extends into their dendrites. Altogether, inhibitory interneurons globally display distinct visual features which reflect their retinal input specialization and are ideally suited to perform feature-selective inhibition.

Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling?

Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.