Perceptography unveils the causal contribution of inferior temporal cortex to visual perception.

Neurons in the inferotemporal (IT) cortex respond selectively to complex visual features, implying their role in object perception. However, perception is subjective and cannot be read out from neural responses; thus, bridging the causal gap between neural activity and perception demands independent characterization of perception. Historically, though, the complexity of the perceptual alterations induced by artificial stimulation of IT cortex has rendered them impossible to quantify. To address this old problem, we tasked male macaque monkeys to detect and report optical impulses delivered to their IT cortex. Combining machine learning with high-throughput behavioral optogenetics, we generated complex and highly specific images that were hard for the animal to distinguish from the state of being cortically stimulated. These images, named "perceptograms" for the first time, reveal and depict the contents of the complex hallucinatory percepts induced by local neural perturbation in IT cortex. Furthermore, we found that the nature and magnitude of these hallucinations highly depend on concurrent visual input, stimulation location, and intensity. Objective characterization of stimulation-induced perceptual events opens the door to developing a mechanistic theory of visual perception. Further, it enables us to make better visual prosthetic devices and gain a greater understanding of visual hallucinations in mental disorders.

Inactivation of face-selective neurons alters eye movements when free viewing faces.

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face-encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of the inferior temporal (IT) cortex in two macaque monkeys using an functional magnetic resonance imaging (fMRI) localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face-selective neurons altered the pattern of eye movements on faces: The monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements.

Behavioral optogenetics in nonhuman primates; a psychological perspective.

Optogenetics has been a promising and developing technology in systems neuroscience throughout the past decade. It has been difficult though to reliably establish the potential behavioral effects of optogenetic perturbation of the neural activity in nonhuman primates. This poses a challenge on the future of optogenetics in humans as the concepts and technology need to be developed in nonhuman primates first. Here, I briefly summarize the viable approaches taken to improve nonhuman primate behavioral optogenetics, then focus on one approach: improvements in the measurement of behavior. I bring examples from visual behavior and show how the choice of method of measurement might conceal large behavioral effects. I will then discuss the "cortical perturbation detection" task in detail as an example of a sensitive task that can record the behavioral effects of optogenetic cortical stimulation with high fidelity. Finally, encouraged by the rich scientific landscape ahead of behavioral optogenetics, I invite technology developers to improve the chronically implantable devices designed for simultaneous neural recording and optogenetic intervention in nonhuman primates.

Inactivation of face selective neurons alters eye movements when free viewing faces.

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of inferior temporal (IT) cortex in two macaque monkeys using an fMRI localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face selective neurons altered the pattern of eye movements on faces: the monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements. Significance: It has been shown, for more than half a century, that eye movements follow distinctive patterns when free viewing faces. This suggests causal involvement of the face-encoding visual neurons in the eye movements. However, the literature is scant of evidence for this possibility and has focused mostly on the link between low-level image saliency and eye movements. Here, for the first time, we bring causal evidence showing how face-selective neurons in inferior temporal cortex inform and steer eye movements when free viewing faces.

Surgical Procedure for Implantation of Opto-Array in Nonhuman Primates.

Optogenetics allows precise temporal control of neuronal activity in the brain. Engineered viral vectors are routinely used to transduce neurons with light-sensitive opsins. However, reliable virus injection and light delivery in animals with large brains, such as nonhuman primates, has proven challenging. The Opto-Array is a novel yet simple device that is used to deliver light to extended regions of the cortex surface for high-throughput behavioral optogenetics in large brains. Here we present protocols for surgical delivery of virus (Basic Protocol 1) and implantation of the Opto-Array (Basic Protocol 2) in two separate surgeries in a rhesus monkey's inferior temporal cortex. As a proof of concept, we measured the behavioral performance of an animal detecting cortical optogenetic stimulations (Basic Protocol 3) with different illumination power and duration using the Opto-Array. The animal was able to detect the optogenetic stimulation for all tested illumination powers and durations. Regression analysis also showed both power and duration of illumination significantly modulate the detectability of the optogenetic stimulation. The outcome of this approach is superior to the standard practice of injecting and recording through a chamber for large areas of the cortex surface. Moreover, the chronic nature of the Opto-Array allows perturbation of neuronal activity of the same site across multiple sessions because it is highly stable; thus, data can be pooled over months. The detailed surgical method presented here makes it possible to use optogenetics to modulate neuronal activity across large regions of the cortex surface in the nonhuman primate brain. This method also lays the groundwork for future attempts to use optogenetics to restore vision in humans. © 2023 The Authors. Current Protocols published by Wiley Periodicals LLC. Basic Protocol 1: Virus injection surgery Basic Protocol 2: Opto-Array implantation surgery Basic Protocol 3: Cortical Perturbation Detection (CPD) task behavioral training.

Image-dependence of the detectability of optogenetic stimulation in macaque inferotemporal cortex.

Artificial activation of neurons in early visual areas induces perception of simple visual flashes.1,2 Accordingly, stimulation in high-level visual cortices is expected to induce perception of complex features.3,4 However, results from studies in human patients challenge this expectation. Stimulation rarely induces any detectable visual event, and never a complex one, in human subjects with closed eyes.2 Stimulation of the face-selective cortex in a human patient led to remarkable hallucinations only while the subject was looking at faces.5 In contrast, stimulations of color- and face-selective sites evoke notable hallucinations independent of the object being viewed.6 These anecdotal observations suggest that stimulation of high-level visual cortex can evoke perception of complex visual features, but these effects depend on the availability and content of visual input. In this study, we introduce a novel psychophysical task to systematically investigate characteristics of the perceptual events evoked by optogenetic stimulation of macaque inferior temporal (IT) cortex. We trained macaque monkeys to detect and report optogenetic impulses delivered to their IT cortices7,8,9 while holding fixation on object images. In a series of experiments, we show that detection of cortical stimulation is highly dependent on the choice of images presented to the eyes and it is most difficult when fixating on a blank screen. These findings suggest that optogenetic stimulation of high-level visual cortex results in easily detectable distortions of the concurrent contents of vision.

Behavioral detectability of optogenetic stimulation of inferior temporal cortex varies with the size of concurrently viewed objects.

We have previously demonstrated that macaque monkeys can behaviorally detect a subtle optogenetic impulse delivered to their inferior temporal (IT) cortex. We have also shown that the ability to detect the cortical stimulation impulse varies depending on some characteristics of the visual images viewed at the time of brain stimulation, revealing the visual nature of the perceptual events induced by stimulation of the IT cortex. Here we systematically studied the effect of the size of viewed objects on behavioral detectability of optogenetic stimulation of the central IT cortex. Surprisingly, we found that behavioral detection of the same optogenetic impulse highly varies with the size of the viewed object images. Reduction of the object size in four steps from 8 to 1 degree of visual angle significantly decreased detection performance. These results show that identical stimulation impulses delivered to the same neural population induce variable perceptual events depending on the mere size of the objects viewed at the time of brain stimulation.

Chronically implantable LED arrays for behavioral optogenetics in primates.

Optogenetic methods have been widely used in rodent brains, but remain relatively under-developed for nonhuman primates such as rhesus macaques, an animal model with a large brain expressing sophisticated sensory, motor and cognitive behaviors. To address challenges in behavioral optogenetics in large brains, we developed Opto-Array, a chronically implantable array of light-emitting diodes for high-throughput optogenetic perturbation. We demonstrated that optogenetic silencing in the macaque primary visual cortex with the help of the Opto-Array results in reliable retinotopic visual deficits in a luminance discrimination task. We separately confirmed that Opto-Array illumination results in local neural silencing, and that behavioral effects are not due to tissue heating. These results demonstrate the effectiveness of the Opto-Array for behavioral optogenetic applications in large brains.

Neurophysiology: The Three-Dimensional Building Blocks of Object Vision.

A new study of the macaque visual cortex has revealed that visual area V4 performs substantial analysis of solid shape structure. The findings draw new attention to the embedding of local three-dimensional shape analysis into the early cortical stages of visual processing.

Richer color vocabulary is associated with better color memory but not color perception.

The potential interaction between color naming and psychophysical color recognition has been historically debated. To study this interaction, here we utilized two approaches based on individual differences in color naming and variation of color name density along the color wheel. We tested a pool of Persian speaking subjects with a simple color matching task under two conditions: perceptual and memory-based matching. We also asked subjects to freely name 100 evenly sampled hues along the color wheel. We found that, individuals who possess more names to describe the color wheel have a strong edge in color memorization over those with fewer names. Nevertheless, having more or fewer color names was not related to the subjects' performance in perceptual color matching. We also calculated the density of color names along the color wheel and observed that parts of the color wheel with higher density of color names are held in memory more accurately. However, similar to the case of individual differences, the density of color names along the wheel did not show any correlation with perceptual color matching performance. Our results demonstrate a strong link between color naming and color memorization both across different individuals and different parts of the color wheel. These results also show that low-level perceptual color matching is not related to color naming, suggesting that the variation in color naming-among the individuals and across the color wheel-is neither the cause nor the effect of variation in low-level color perception.

An Open Resource for Non-human Primate Optogenetics.

Optogenetics has revolutionized neuroscience in small laboratory animals, but its effect on animal models more closely related to humans, such as non-human primates (NHPs), has been mixed. To make evidence-based decisions in primate optogenetics, the scientific community would benefit from a centralized database listing all attempts, successful and unsuccessful, of using optogenetics in the primate brain. We contacted members of the community to ask for their contributions to an open science initiative. As of this writing, 45 laboratories around the world contributed more than 1,000 injection experiments, including precise details regarding their methods and outcomes. Of those entries, more than half had not been published. The resource is free for everyone to consult and contribute to on the Open Science Framework website. Here we review some of the insights from this initial release of the database and discuss methodological considerations to improve the success of optogenetic experiments in NHPs.

Paradoxical impact of memory on color appearance of faces.

What is color vision for? Here we compared the extent to which memory modulates color appearance of objects and faces. Participants matched the colors of stimuli illuminated by low-pressure sodium light, which renders scenes monochromatic. Matches for fruit were not predicted by stimulus identity. In contrast, matches for faces were predictable, but surprising: faces appeared green and looked sick. The paradoxical face-color percept could be explained by a Bayesian observer model constrained by efficient coding. The color-matching data suggest that the face-color prior is established by visual signals arising from the recently evolved L-M cone system, not the older S-cone channel. Taken together, the results show that when retinal mechanisms of color vision are impaired, the impact of memory on color perception is greatest for face color, supporting the idea that trichromatic color plays an important role in social communication.

Insight into the Neuron's Insight.

In this issue, Ponce and colleagues use a generative closed-loop system to evolve synthetic images to explore the response properties of neurons in the inferior temporal cortex of non-human primates. The results reveal an unbiased assessment of feature selectivity in a high-level visual area involved in object recognition.

Navigating the Neural Space in Search of the Neural Code.

The advent of powerful perturbation tools, such as optogenetics, has created new frontiers for probing causal dependencies in neural and behavioral states. These approaches have significantly enhanced the ability to characterize the contribution of different cells and circuits to neural function in health and disease. They have shifted the emphasis of research toward causal interrogations and increased the demand for more precise and powerful tools to control and manipulate neural activity. Here, we clarify the conditions under which measurements and perturbations support causal inferences. We note that the brain functions at multiple scales and that causal dependencies may be best inferred with perturbation tools that interface with the system at the appropriate scale. Finally, we develop a geometric framework to facilitate the interpretation of causal experiments when brain perturbations do or do not respect the intrinsic patterns of brain activity. We describe the challenges and opportunities of applying perturbations in the presence of dynamics, and we close with a general perspective on navigating the activity space of neurons in the search for neural codes.

Optogenetic and pharmacological suppression of spatial clusters of face neurons reveal their causal role in face gender discrimination.

Neurons that respond more to images of faces over nonface objects were identified in the inferior temporal (IT) cortex of primates three decades ago. Although it is hypothesized that perceptual discrimination between faces depends on the neural activity of IT subregions enriched with "face neurons," such a causal link has not been directly established. Here, using optogenetic and pharmacological methods, we reversibly suppressed the neural activity in small subregions of IT cortex of macaque monkeys performing a facial gender-discrimination task. Each type of intervention independently demonstrated that suppression of IT subregions enriched in face neurons induced a contralateral deficit in face gender-discrimination behavior. The same neural suppression of other IT subregions produced no detectable change in behavior. These results establish a causal link between the neural activity in IT face neuron subregions and face gender-discrimination behavior. Also, the demonstration that brief neural suppression of specific spatial subregions of IT induces behavioral effects opens the door for applying the technical advantages of optogenetics to a systematic attack on the causal relationship between IT cortex and high-level visual perception.

Neural Mechanisms Underlying Visual Object Recognition.

Invariant visual object recognition and the underlying neural representations are fundamental to higher-level human cognition. To understand these neural underpinnings, we combine human and monkey psychophysics, large-scale neurophysiology, neural perturbation methods, and computational modeling to construct falsifiable, predictive models that aim to fully account for the neural encoding and decoding processes that underlie visual object recognition. A predictive encoding model must minimally describe the transformation of the retinal image to population patterns of neural activity along the entire cortical ventral stream of visual processing and must accurately predict the responses to any retinal image. A predictive decoding model must minimally describe the transformation from those population patterns of neural activity to observed object recognition behavior (i.e., subject reports), and, given that population pattern of activity, it must accurately predict behavior for any object recognition task. To date, we have focused on core object recognition-a remarkable behavior that is accomplished with image viewing durations of <200 msec. Our work thus far reveals that the neural encoding process is reasonably well explained by a largely feed-forward, highly complex, multistaged nonlinear neural network-the current best neuronal simulation models predict approximately one-half of the relevant neuronal response variance across the highest levels of the ventral stream (areas V4 and IT). Remarkably, however, the decoding process from IT to behavior for all object recognition tasks tested thus far is very accurately predicted by simple direct linear conversion of the inferior temporal neural population state to behavior choice. We have recently examined the behavioral consequences of direct suppression of IT neural activity using pharmacological and optogenetic methods and find them to be well-explained by the same linear decoding model.