Differential clustering of visual and choice- and saccade-related activity in macaque V3A and CIP.

Neurons in sensory and motor cortices tend to aggregate in clusters with similar functional properties. Within the primate dorsal ("where") pathway, an important interface between three-dimensional (3-D) visual processing and motor-related functions consists of two hierarchically organized areas: V3A and the caudal intraparietal (CIP) area. In these areas, 3-D visual information, choice-related activity, and saccade-related activity converge, often at the single-neuron level. Characterizing the clustering of functional properties in areas with mixed selectivity, such as these, may help reveal organizational principles that support sensorimotor transformations. Here we quantified the clustering of visual feature selectivity, choice-related activity, and saccade-related activity by performing correlational and parametric comparisons of the responses of well-isolated, simultaneously recorded neurons in macaque monkeys. Each functional domain showed statistically significant clustering in both areas. However, there were also domain-specific differences in the strength of clustering across the areas. Visual feature selectivity and saccade-related activity were more strongly clustered in V3A than in CIP. In contrast, choice-related activity was more strongly clustered in CIP than in V3A. These differences in clustering may reflect the areas' roles in sensorimotor processing. Stronger clustering of visual and saccade-related activity in V3A may reflect a greater role in within-domain processing, as opposed to cross-domain synthesis. In contrast, stronger clustering of choice-related activity in CIP may reflect a greater role in synthesizing information across functional domains to bridge perception and action.NEW & NOTEWORTHY The occipital and parietal cortices of macaque monkeys are bridged by hierarchically organized areas V3A and CIP. These areas support 3-D visual transformations, carry choice-related activity during 3-D perceptual tasks, and possess saccade-related activity. This study quantifies the functional clustering of neuronal response properties within V3A and CIP for each of these domains. The findings reveal domain-specific cross-area differences in clustering that may reflect the areas' roles in sensorimotor processing.

Neuronal Representations Supporting Three-Dimensional Vision in Nonhuman Primates.

The visual system must reconstruct the dynamic, three-dimensional (3D) world from ambiguous two-dimensional (2D) retinal images. In this review, we synthesize current literature on how the visual system of nonhuman primates performs this transformation through multiple channels within the classically defined dorsal (where) and ventral (what) pathways. Each of these channels is specialized for processing different 3D features (e.g., the shape, orientation, or motion of objects, or the larger scene structure). Despite the common goal of 3D reconstruction, neurocomputational differences between the channels impose distinct information-limiting constraints on perception. Convergent evidence further points to the little-studied area V3A as a potential branchpoint from which multiple 3D-fugal processing channels diverge. We speculate that the expansion of V3A in humans may have supported the emergence of advanced 3D spatial reasoning skills. Lastly, we discuss future directions for exploring 3D information transmission across brain areas and experimental approaches that can further advance the understanding of 3D vision.

Parallel processing, hierarchical transformations, and sensorimotor associations along the 'where' pathway.

Visually guided behaviors require the brain to transform ambiguous retinal images into object-level spatial representations and implement sensorimotor transformations. These processes are supported by the dorsal 'where' pathway. However, the specific functional contributions of areas along this pathway remain elusive due in part to methodological differences across studies. We previously showed that macaque caudal intraparietal (CIP) area neurons possess robust 3D visual representations, carry choice- and saccade-related activity, and exhibit experience-dependent sensorimotor associations (Chang et al., 2020b). Here, we used a common experimental design to reveal parallel processing, hierarchical transformations, and the formation of sensorimotor associations along the 'where' pathway by extending the investigation to V3A, a major feedforward input to CIP. Higher-level 3D representations and choice-related activity were more prevalent in CIP than V3A. Both areas contained saccade-related activity that predicted the direction/timing of eye movements. Intriguingly, the time course of saccade-related activity in CIP aligned with the temporally integrated V3A output. Sensorimotor associations between 3D orientation and saccade direction preferences were stronger in CIP than V3A, and moderated by choice signals in both areas. Together, the results explicate parallel representations, hierarchical transformations, and functional associations of visual and saccade-related signals at a key juncture in the 'where' pathway.

Neural Correlates of Multisensory Integration for Feedback Stabilization of the Wrist.

Robust control of action relies on the ability to perceive, integrate, and act on information from multiple sensory modalities including vision and proprioception. How does the brain combine sensory information to regulate ongoing mechanical interactions between the body and its physical environment? Some behavioral studies suggest that the rules governing multisensory integration for action may differ from the maximum likelihood estimation rules that appear to govern multisensory integration for many perceptual tasks. We used functional magnetic resonance (MR) imaging techniques, a MR-compatible robot, and a multisensory feedback control task to test that hypothesis by investigating how neural mechanisms involved in regulating hand position against mechanical perturbation respond to the presence and fidelity of visual and proprioceptive information. Healthy human subjects rested supine in a MR scanner and stabilized their wrist against constant or pseudo-random torque perturbations imposed by the robot. These two stabilization tasks were performed under three visual feedback conditions: "No-vision": Subjects had to rely solely on proprioceptive feedback; "true-vision": visual cursor and hand motions were congruent; and "random-vision": cursor and hand motions were uncorrelated in time. Behaviorally, performance errors accumulated more quickly during trials wherein visual feedback was absent or incongruous. We analyzed blood-oxygenation level-dependent (BOLD) signal fluctuations to compare task-related activations in a cerebello-thalamo-cortical neural circuit previously linked with feedback stabilization of the hand. Activation in this network varied systematically depending on the presence and fidelity of visual feedback of task performance. Addition of task related visual information caused activations in the cerebello-thalamo-cortical network to expand into neighboring brain regions. Specific loci and intensity of expanded activity depended on the fidelity of visual feedback. Remarkably, BOLD signal fluctuations within these regions correlated strongly with the time series of proprioceptive errors-but not visual errors-when the fidelity of visual feedback was poor, even though visual and hand motions had similar variability characteristics. These results provide insight into the neural control of the body's physical interactions with its environment, rejecting the standard Gaussian cue combination model of multisensory integration in favor of models that account for causal structure in the sensory feedback.

MRI Compatible, Customizable, and 3D-Printable Microdrive for Neuroscience Research.

The effective connectivity of brain networks can be assessed using functional magnetic resonance imaging (fMRI) to quantify the effects of local electrical microstimulation (EM) on distributed neuronal activity. The delivery of EM to specific brain regions, particularly with layer specificity, requires MRI compatible equipment that provides fine control of a stimulating electrode's position within the brain while minimizing imaging artifacts. To this end, we developed a microdrive made entirely of MRI compatible materials. The microdrive uses an integrated penetration grid to guide electrodes and relies on a microdrilling technique to eliminate the need for large craniotomies, further reducing implant maintenance and image distortions. The penetration grid additionally serves as a built-in MRI marker, providing a visible fiducial reference for estimating probe trajectories. Following the initial implant procedure, these features allow for multiple electrodes to be inserted, removed, and repositioned with minimal effort, using a screw-type actuator. To validate the design of the microdrive, we conducted an EM-coupled fMRI study with a male macaque monkey. The results verified that the microdrive can be used to deliver EM during MRI procedures with minimal imaging artifacts, even within a 7 Tesla (7T) environment. Future applications of the microdrive include neuronal recordings and targeted drug delivery. We provide computer aided design (CAD) templates and a parts list for modifying and fabricating the microdrive for specific research needs. These designs provide a convenient, cost-effective approach to fabricating MRI compatible microdrives for neuroscience research.

Functional links between sensory representations, choice activity, and sensorimotor associations in parietal cortex.

Three-dimensional (3D) representations of the environment are often critical for selecting actions that achieve desired goals. The success of these goal-directed actions relies on 3D sensorimotor transformations that are experience-dependent. Here we investigated the relationships between the robustness of 3D visual representations, choice-related activity, and motor-related activity in parietal cortex. Macaque monkeys performed an eight-alternative 3D orientation discrimination task and a visually guided saccade task while we recorded from the caudal intraparietal area using laminar probes. We found that neurons with more robust 3D visual representations preferentially carried choice-related activity. Following the onset of choice-related activity, the robustness of the 3D representations further increased for those neurons. We additionally found that 3D orientation and saccade direction preferences aligned, particularly for neurons with choice-related activity, reflecting an experience-dependent sensorimotor association. These findings reveal previously unrecognized links between the fidelity of ecologically relevant object representations, choice-related activity, and motor-related activity.

Optimized but Not Maximized Cue Integration for 3D Visual Perception.

Reconstructing three-dimensional (3D) scenes from two-dimensional (2D) retinal images is an ill-posed problem. Despite this, 3D perception of the world based on 2D retinal images is seemingly accurate and precise. The integration of distinct visual cues is essential for robust 3D perception in humans, but it is unclear whether this is true for non-human primates (NHPs). Here, we assessed 3D perception in macaque monkeys using a planar surface orientation discrimination task. Perception was accurate across a wide range of spatial poses (orientations and distances), but precision was highly dependent on the plane's pose. The monkeys achieved robust 3D perception by dynamically reweighting the integration of stereoscopic and perspective cues according to their pose-dependent reliabilities. Errors in performance could be explained by a prior resembling the 3D orientation statistics of natural scenes. We used neural network simulations based on 3D orientation-selective neurons recorded from the same monkeys to assess how neural computation might constrain perception. The perceptual data were consistent with a model in which the responses of two independent neuronal populations representing stereoscopic cues and perspective cues (with perspective signals from the two eyes combined using nonlinear canonical computations) were optimally integrated through linear summation. Perception of combined-cue stimuli was optimal given this architecture. However, an alternative architecture in which stereoscopic cues, left eye perspective cues, and right eye perspective cues were represented by three independent populations yielded two times greater precision than the monkeys. This result suggests that, due to canonical computations, cue integration for 3D perception is optimized but not maximized.

Real-time experimental control using network-based parallel processing.

Modern neuroscience research often requires the coordination of multiple processes such as stimulus generation, real-time experimental control, as well as behavioral and neural measurements. The technical demands required to simultaneously manage these processes with high temporal fidelity is a barrier that limits the number of labs performing such work. Here we present an open-source, network-based parallel processing framework that lowers this barrier. The Real-Time Experimental Control with Graphical User Interface (REC-GUI) framework offers multiple advantages: (i) a modular design that is agnostic to coding language(s) and operating system(s) to maximize experimental flexibility and minimize researcher effort, (ii) simple interfacing to connect multiple measurement and recording devices, (iii) high temporal fidelity by dividing task demands across CPUs, and (iv) real-time control using a fully customizable and intuitive GUI. We present applications for human, non-human primate, and rodent studies which collectively demonstrate that the REC-GUI framework facilitates technically demanding, behavior-contingent neuroscience research. Editorial note: This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (see decision letter).