How Shape Perception Works, in Two Dimensions and Three Dimensions.

The ventral visual pathway transforms retinal images into neural representations that support object understanding, including exquisite appreciation of precise 2D pattern shape and 3D volumetric shape. We articulate a framework for understanding the goals of this transformation and how they are achieved by neural coding at successive ventral pathway stages. The critical goals are (a) radical compression to make shape information communicable across axonal bundles and storable in memory, (b) explicit coding to make shape information easily readable by the rest of the brain and thus accessible for cognition and behavioral control, and (c) representational stability to maintain consistent perception across highly variable viewing conditions. We describe how each transformational step in ventral pathway vision serves one or more of these goals. This three-goal framework unifies discoveries about ventral shape processing into a neural explanation for our remarkable experience of shape as a vivid, richly detailed aspect of the natural world.

The Cousa objective: a long-working distance air objective for multiphoton imaging in vivo.

Multiphoton microscopy can resolve fluorescent structures and dynamics deep in scattering tissue and has transformed neural imaging, but applying this technique in vivo can be limited by the mechanical and optical constraints of conventional objectives. Short working distance objectives can collide with compact surgical windows or other instrumentation and preclude imaging. Here we present an ultra-long working distance (20 mm) air objective called the Cousa objective. It is optimized for performance across multiphoton imaging wavelengths, offers a more than 4 mm2 field of view with submicrometer lateral resolution and is compatible with commonly used multiphoton imaging systems. A novel mechanical design, wider than typical microscope objectives, enabled this combination of specifications. We share the full optical prescription, and report performance including in vivo two-photon and three-photon imaging in an array of species and preparations, including nonhuman primates. The Cousa objective can enable a range of experiments in neuroscience and beyond.

Object representation in a gravitational reference frame.

When your head tilts laterally, as in sports, reaching, and resting, your eyes counterrotate less than 20%, and thus eye images rotate, over a total range of about 180°. Yet, the world appears stable and vision remains normal. We discovered a neural strategy for rotational stability in anterior inferotemporal cortex (IT), the final stage of object vision in primates. We measured object orientation tuning of IT neurons in macaque monkeys tilted +25 and -25° laterally, producing ~40° difference in retinal image orientation. Among IT neurons with consistent object orientation tuning, 63% remained stable with respect to gravity across tilts. Gravitational tuning depended on vestibular/somatosensory but also visual cues, consistent with previous evidence that IT processes scene cues for gravity's orientation. In addition to stability across image rotations, an internal gravitational reference frame is important for physical understanding of a world where object position, posture, structure, shape, movement, and behavior interact critically with gravity.

Cell-specific regulation of gene expression using splicing-dependent frameshifting.

Precise and reliable cell-specific gene delivery remains technically challenging. Here we report a splicing-based approach for controlling gene expression whereby separate translational reading frames are coupled to the inclusion or exclusion of mutated, frameshifting cell-specific alternative exons. Candidate exons are identified by analyzing thousands of publicly available RNA sequencing datasets and filtering by cell specificity, conservation, and local intron length. This method, which we denote splicing-linked expression design (SLED), can be combined in a Boolean manner with existing techniques such as minipromoters and viral capsids. SLED can use strong constitutive promoters, without sacrificing precision, by decoupling the tradeoff between promoter strength and selectivity. AAV-packaged SLED vectors can selectively deliver fluorescent reporters and calcium indicators to various neuronal subtypes in vivo. We also demonstrate gene therapy utility by creating SLED vectors that can target PRPH2 and SF3B1 mutations. The flexibility of SLED technology enables creative avenues for basic and translational research.

Development of visual motion integration involves coordination of multiple cortical stages.

A central feature of cortical function is hierarchical processing of information. Little is currently known about how cortical processing cascades develop. Here, we investigate the joint development of two nodes of the ferret's visual motion pathway, primary visual cortex (V1), and higher-level area PSS. In adult animals, motion processing transitions from local to global computations between these areas. We now show that PSS global motion signals emerge a week after the development of V1 and PSS direction selectivity. Crucially, V1 responses to more complex motion stimuli change in parallel, in a manner consistent with supporting increased PSS motion integration. At the same time, these V1 responses depend on feedback from PSS. Our findings suggest that development does not just proceed in parallel in different visual areas, it is coordinated across network nodes. This has important implications for understanding how visual experience and developmental disorders can influence the developing visual system.

Early Emergence of Solid Shape Coding in Natural and Deep Network Vision.

Area V4 is the first object-specific processing stage in the ventral visual pathway, just as area MT is the first motion-specific processing stage in the dorsal pathway. For almost 50 years, coding of object shape in V4 has been studied and conceived in terms of flat pattern processing, given its early position in the transformation of 2D visual images. Here, however, in awake monkey recording experiments, we found that roughly half of V4 neurons are more tuned and responsive to solid, 3D shape-in-depth, as conveyed by shading, specularity, reflection, refraction, or disparity cues in images. Using 2-photon functional microscopy, we found that flat- and solid-preferring neurons were segregated into separate modules across the surface of area V4. These findings should impact early shape-processing theories and models, which have focused on 2D pattern processing. In fact, our analyses of early object processing in AlexNet, a standard visual deep network, revealed a similar distribution of sensitivities to flat and solid shape in layer 3. Early processing of solid shape, in parallel with flat shape, could represent a computational advantage discovered by both primate brain evolution and deep-network training.

Advanced Circuit and Cellular Imaging Methods in Nonhuman Primates.

Novel genetically encoded tools and advanced microscopy methods have revolutionized neural circuit analyses in insects and rodents over the last two decades. Whereas numerous technical hurdles originally barred these methodologies from success in nonhuman primates (NHPs), current research has started to overcome those barriers. In some cases, methodological advances developed with NHPs have even surpassed their precursors. One such advance includes new ultra-large imaging windows on NHP cortex, which are larger than the entire rodent brain and allow analysis unprecedented ultra-large-scale circuits. NHP imaging chambers now remain patent for periods longer than a mouse's lifespan, allowing for long-term all-optical interrogation of identified circuits and neurons over timeframes that are relevant to human cognitive development. Here we present some recent imaging advances brought forth by research teams using macaques and marmosets. These include technical developments in optogenetics; voltage-, calcium- and glutamate-sensitive dye imaging; two-photon and wide-field optical imaging; viral delivery; and genetic expression of indicators and light-activated proteins that result in the visualization of tens of thousands of identified cortical neurons in NHPs. We describe a subset of the many recent advances in circuit and cellular imaging tools in NHPs focusing here primarily on the research presented during the corresponding mini-symposium at the 2019 Society for Neuroscience annual meeting.

Visual Motion and Form Integration in the Behaving Ferret.

Ferrets have become a standard animal model for the development of early visual stages. Less is known about higher-level vision in ferrets, both during development and in adulthood. Here, as a step towards establishing higher-level vision research in ferrets, we used behavioral experiments to test the motion and form integration capacity of adult ferrets. Motion integration was assessed by training ferrets to discriminate random dot kinematograms (RDK) based on their direction. Task difficulty was varied systematically by changing RDK coherence levels, which allowed the measurement of motion integration thresholds. Form integration was measured analogously by training ferrets to discriminate linear Glass patterns of varying coherence levels based on their orientation. In all experiments, ferrets proved to be good psychophysical subjects that performed tasks reliably. Crucially, the behavioral data showed clear evidence of perceptual motion and form integration. In the monkey, motion and form integration are usually associated with processes occurring in higher-level visual areas. In a second set of experiments, we therefore tested whether PSS, a higher-level motion area in the ferret, could similarly support motion integration behavior in this species. To this end, we measured responses of PSS neurons to RDK of different coherence levels. Indeed, neurometric functions for PSS were in good agreement with the behaviorally derived psychometric functions. In conclusion, our experiments demonstrate that ferrets are well suited for higher-level vision research.

Ferrets as a Model for Higher-Level Visual Motion Processing.

Ferrets are a major developmental animal model due to their early parturition. Here we show for the first time that ferrets could be used to study development of higher-level visual processes previously identified in primates. In primates, complex motion processing involves primary visual cortex (V1), which generates local motion signals, and higher-level visual area MT, which integrates these signals over more global spatial regions. Our data show similar transformations in motion signals between ferret V1 and higher-level visual area PSS, located in the posterior bank of the suprasylvian sulcus. We found that PSS neurons, like MT neurons, were tuned for stimulus motion and showed strong suppression between opposing direction inputs. Most strikingly, PSS, like MT, exhibited robust global motion signals when tested with coherent plaids-the classic test for motion integration across multiple moving elements. These PSS responses were described well by computational models developed for MT. Our findings establish the ferret as a strong animal model for development of higher-level visual processing.

Efficient Receptive Field Tiling in Primate V1.

The primary visual cortex (V1) encodes a diverse set of visual features, including orientation, ocular dominance (OD), and spatial frequency (SF), whose joint organization must be precisely structured to optimize coverage within the retinotopic map. Prior experiments have only identified efficient coverage based on orthogonal maps. Here we used two-photon calcium imaging to reveal an alternative arrangement for OD and SF maps in macaque V1; their gradients run parallel but with unique spatial periods, whereby low-SF regions coincide with monocular regions. Next we mapped receptive fields and found surprisingly precise micro-retinotopy that yields a smaller point-image and requires more efficient inter-map geometry, thus underscoring the significance of map relationships. While smooth retinotopy is constraining, studies suggest that it improves both wiring economy and the V1 population code read downstream. Altogether, these data indicate that connectivity within V1 is finely tuned and precise at the level of individual neurons.

Building maps from maps in primary visual cortex.

Neurons in the visual system respond to more complex and holistic features at each new stage of processing. Often, these features are organized into continuous maps. Could there be a fundamental link between continuous maps and functional hierarchies? Here, we review recent studies regarding V1 maps providing some of the most noteworthy advances in our understanding of how and why maps exist. In particular, we focus on the common theme that some maps are inherited from the input of parallel pathways, which are then intimately linked to the emergence of new functional properties and their corresponding maps. These results on V1 maps may prove to be a unifying framework for hierarchical representations in the visual cortex.

Orthogonal micro-organization of orientation and spatial frequency in primate primary visual cortex.

Orientation and spatial frequency tuning are highly salient properties of neurons in primary visual cortex (V1). The combined organization of these particular tuning properties in the cortical space will strongly shape the V1 population response to different visual inputs, yet it is poorly understood. In this study, we used two-photon imaging in macaque monkey V1 to demonstrate the three-dimensional cell-by-cell layout of both spatial frequency and orientation tuning. We first found that spatial frequency tuning was organized into highly structured maps that remained consistent across the depth of layer II/III, similarly to orientation tuning. Next, we found that orientation and spatial frequency maps were intimately related at the fine spatial scale observed with two-photon imaging. Not only did the map gradients tend notably toward orthogonality, but they also co-varied negatively from cell to cell at the spatial scale of cortical columns.

Viral vector-based reversible neuronal inactivation and behavioral manipulation in the macaque monkey.

Viral vectors are promising tools for the dissection of neural circuits. In principle, they can manipulate neurons at a level of specificity not otherwise achievable. While many studies have used viral vector-based approaches in the rodent brain, only a few have employed this technique in the non-human primate, despite the importance of this animal model for neuroscience research. Here, we report evidence that a viral vector-based approach can be used to manipulate a monkey's behavior in a task. For this purpose, we used the allatostatin receptor/allatostatin (AlstR/AL) system, which has previously been shown to allow inactivation of neurons in vivo. The AlstR was expressed in neurons in monkey V1 by injection of an adeno-associated virus 1 (AAV1) vector. Two monkeys were trained in a detection task, in which they had to make a saccade to a faint peripheral target. Injection of AL caused a retinotopic deficit in the detection task in one monkey. Specifically, the monkey showed marked impairment for detection targets placed at the visual field location represented at the virus injection site, but not for targets shown elsewhere. We confirmed that these deficits indeed were due to the interaction of AlstR and AL by injecting saline, or AL at a V1 location without AlstR expression. Post-mortem histology confirmed AlstR expression in this monkey. We failed to replicate the behavioral results in a second monkey, as AL injection did not impair the second monkey's performance in the detection task. However, post-mortem histology revealed a very low level of AlstR expression in this monkey. Our results demonstrate that viral vector-based approaches can produce effects strong enough to influence a monkey's performance in a behavioral task, supporting the further development of this approach for studying how neuronal circuits control complex behaviors in non-human primates.

Nonlinearity of two-photon Ca2+ imaging yields distorted measurements of tuning for V1 neuronal populations.

We studied the relative accuracy of drifting gratings and noise stimuli for functionally characterizing neural populations using two-photon calcium imaging. Calcium imaging has the potential to distort measurements due to nonlinearity in the conversion from spikes to observed fluorescence. We demonstrate a dramatic impact of fluorescence saturation on functional measurements in ferret V1 by showing that responses to drifting gratings strongly violate contrast invariance of orientation tuning, a fundamental property of the spike rates. The observed relationship is consistent with saturation that clips the high-contrast tuning curve peaks by ∼40%. The nonlinearity was also apparent in mouse V1 responses to drifting gratings, but not as strong as in the ferret. Contrast invariance holds, however, for tuning curves measured with a randomized grating stimulus. This finding is consistent with prior work showing that the linear portion of a linear-nonlinear system can be recovered with reverse correlation. Furthermore, we demonstrate that a noise stimulus is more effective at keeping spike rates in the linear operating regime of a saturating nonlinearity, which both maximizes signal-to-noise ratios and simplifies the recovery of fast spike dynamics from slow calcium transients. Finally, we uncover spatiotemporal receptive fields by removing the nonlinearity and slow calcium transient from a model of fluorescence generation, which allowed us to observe dynamic sharpening of orientation tuning. We conclude that for two-photon recordings it is imperative that one considers the nonlinear distortion when designing stimuli and interpreting results, especially in sensory areas, species, or cell types with high firing rates.

Targeting single neuronal networks for gene expression and cell labeling in vivo.

To understand fine-scale structure and function of single mammalian neuronal networks, we developed and validated a strategy to genetically target and trace monosynaptic inputs to a single neuron in vitro and in vivo. The strategy independently targets a neuron and its presynaptic network for specific gene expression and fine-scale labeling, using single-cell electroporation of DNA to target infection and monosynaptic retrograde spread of a genetically modifiable rabies virus. The technique is highly reliable, with transsynaptic labeling occurring in every electroporated neuron infected by the virus. Targeting single neocortical neuronal networks in vivo, we found clusters of both spiny and aspiny neurons surrounding the electroporated neuron in each case, in addition to intricately labeled distal cortical and subcortical inputs. This technique, broadly applicable for probing and manipulating single neuronal networks with single-cell resolution in vivo, may help shed new light on fundamental mechanisms underlying circuit development and information processing by neuronal networks throughout the brain.

Object features used by humans and monkeys to identify rotated shapes.

Humans and rhesus monkeys can identify shapes that have been rotated in the picture plane. Recognition of rotated shapes can be as efficient as recognition of upright shapes. Here we investigate whether subjects showing view-invariant performance use the same object features to identify upright and rotated versions of a shape. We find marked differences between humans and monkeys. While humans tend to use the same features independent of shape orientation, monkeys use unique features for each orientation. Humans are able to generalize to a greater degree across orientation changes than rhesus monkey observers, who tend to relearn separate problems at each orientation rather than flexibly apply previously learned knowledge to novel problems.

More than a feeling: sensation from cortical stimulation.

Changes in neuronal firing underlie sensation, but how many neurons are needed to perceive these activity shifts? Two new studies in Nature suggest that the experimental modulation of only a few neurons can influence perception.

Object recognition: similar visual strategies of birds and mammals.

Behavioral testing has revealed that pigeons may use the same visual information sources as humans to discriminate between three-dimensional shapes.

Dissociation between local field potentials and spiking activity in macaque inferior temporal cortex reveals diagnosticity-based encoding of complex objects.

Neurons in the inferior temporal (IT) cortex respond selectively to complex objects, and maintain their selectivity despite partial occlusion. However, relatively little is known about how the occlusion of different shape parts influences responses in the IT cortex. Here, we determine experimentally which parts of complex objects monkeys are relying on in a discrimination task. We then study the effect of occlusion of parts with different behavioral relevance on neural responses in the IT cortex at the level of spiking activity and local field potentials (LFPs). For both spiking activity and LFPs, we found that the diagnostic object parts, which were important for behavioral judgments, were preferentially represented in the IT cortex. Our data show that the effects of diagnosticity grew systematically stronger along a posterior-anterior axis for LFPs, but were evenly distributed for single units, suggesting that diagnosticity is first encoded in the posterior IT cortex. Our findings highlight the power of combined analysis of field potentials and spiking activity for mapping structure to computational function in the brain.

Discrimination strategies of humans and rhesus monkeys for complex visual displays.

By learning to discriminate among visual stimuli, human observers can become experts at specific visual tasks. The same is true for Rhesus monkeys, the major animal model of human visual perception. Here, we systematically compare how humans and monkeys solve a simple visual task. We trained humans and monkeys to discriminate between the members of small natural-image sets. We employed the "Bubbles" procedure to determine the stimulus features used by the observers. On average, monkeys used image features drawn from a diagnostic region covering about 7% +/- 2% of the images. Humans were able to use image features drawn from a much larger diagnostic region covering on average 51% +/- 4% of the images. Similarly for the two species, however, about 2% of the image needed to be visible within the diagnostic region on any individual trial for correct performance. We characterize the low-level image properties of the diagnostic regions and discuss individual differences among the monkeys. Our results reveal that monkeys base their behavior on confined image patches and essentially ignore a large fraction of the visual input, whereas humans are able to gather visual information with greater flexibility from large image regions.