Visual sensitivity at the service of action control in posterior parietal cortex.

The posterior parietal cortex (PPC) serves as a crucial hub for the integration of sensory with motor cues related to voluntary actions. Visual input is used in different ways along the dorsomedial and the dorsolateral visual pathways. Here we focus on the dorsomedial pathway and recognize a visual representation at the service of action control. Employing different experimental paradigms applied to behaving monkeys while single neural activity is recorded from the medial PPC (area V6A), we show how plastic visual representation can be, matching the different contexts in which the same object is proposed. We also present data on the exchange between vision and arm actions and highlight how this rich interplay can be used to weight different sensory inputs in order to monitor and correct arm actions online. Indeed, neural activity during reaching or reach-to-grasp actions can be excited or inhibited by visual information, suggesting that the visual perception of action, rather than object recognition, is the most effective factor for area V6A. Also, three-dimensional object shape is encoded dynamically by the neural population, according to the behavioral context of the monkey. Along this line, mirror neuron discharges in V6A indicate the plasticity of visual representation of the graspable objects, that changes according to the context and peaks when the object is the target of one's own action. In other words, object encoding in V6A is a visual encoding for action.

Modulation of reaching by spatial attention.

Attention is needed to perform goal-directed vision-guided movements. We investigated whether the direction of covert attention modulates movement outcomes and dynamics. Right-handed and left-handed volunteers attended to a spatial location while planning a reach toward the same hemifield, the opposite one, or planned a reach without constraining attention. We measured behavioral variables as outcomes of ipsilateral and contralateral reaching and the tangling of behavioral trajectories obtained through principal component analysis as a measure of the dynamics of motor control. We found that the direction of covert attention had significant effects on the dynamics of motor control, specifically during contralateral reaching. Data suggest that motor control was more feedback-driven when attention was directed leftward than when attention was directed rightward or when it was not constrained, irrespectively of handedness. These results may help to better understand the neural bases of asymmetrical neurological diseases like hemispatial neglect.

Convolutional neural networks reveal properties of reach-to-grasp encoding in posterior parietal cortex.

Deep neural networks (DNNs) are widely adopted to decode motor states from both non-invasively and invasively recorded neural signals, e.g., for realizing brain-computer interfaces. However, the neurophysiological interpretation of how DNNs make the decision based on the input neural activity is limitedly addressed, especially when applied to invasively recorded data. This reduces decoder reliability and transparency, and prevents the exploitation of decoders to better comprehend motor neural encoding. Here, we adopted an explainable artificial intelligence approach - based on a convolutional neural network and an explanation technique - to reveal spatial and temporal neural properties of reach-to-grasping from single-neuron recordings of the posterior parietal area V6A. The network was able to accurately decode 5 different grip types, and the explanation technique automatically identified the cells and temporal samples that most influenced the network prediction. Grip encoding in V6A neurons already started at movement preparation, peaking during movement execution. A difference was found within V6A: dorsal V6A neurons progressively encoded more for increasingly advanced grips, while ventral V6A neurons for increasingly rudimentary grips, with both subareas following a linear trend between the amount of grip encoding and the level of grip skills. By revealing the elements of the neural activity most relevant for each grip with no a priori assumptions, our approach supports and advances current knowledge about reach-to-grasp encoding in V6A, and it may represent a general tool able to investigate neural correlates of motor or cognitive tasks (e.g., attention and memory tasks) from single-neuron recordings.

Neural sensitivity to translational self- and object-motion velocities.

The ability to detect and assess world-relative object-motion is a critical computation performed by the visual system. This computation, however, is greatly complicated by the observer's movements, which generate a global pattern of motion on the observer's retina. How the visual system implements this computation is poorly understood. Since we are potentially able to detect a moving object if its motion differs in velocity (or direction) from the expected optic flow generated by our own motion, here we manipulated the relative motion velocity between the observer and the object within a stationary scene as a strategy to test how the brain accomplishes object-motion detection. Specifically, we tested the neural sensitivity of brain regions that are known to respond to egomotion-compatible visual motion (i.e., egomotion areas: cingulate sulcus visual area, posterior cingulate sulcus area, posterior insular cortex [PIC], V6+, V3A, IPSmot/VIP, and MT+) to a combination of different velocities of visually induced translational self- and object-motion within a virtual scene while participants were instructed to detect object-motion. To this aim, we combined individual surface-based brain mapping, task-evoked activity by functional magnetic resonance imaging, and parametric and representational similarity analyses. We found that all the egomotion regions (except area PIC) responded to all the possible combinations of self- and object-motion and were modulated by the self-motion velocity. Interestingly, we found that, among all the egomotion areas, only MT+, V6+, and V3A were further modulated by object-motion velocities, hence reflecting their possible role in discriminating between distinct velocities of self- and object-motion. We suggest that these egomotion regions may be involved in the complex computation required for detecting scene-relative object-motion during self-motion.

rTMS over the human medial parietal cortex impairs online reaching corrections.

Indirect correlational evidence suggests that the posteromedial sector of the human parietal cortex (area hV6A) is involved in reaching corrections. We interfered with hV6A functions using repetitive transcranial magnetic stimulation (rTMS) while healthy participants performed reaching movements and in-flight adjustments of the hand trajectory in presence of unexpected target shifts. rTMS over hV6A specifically altered action reprogramming, causing deviations of the shifted trajectories, particularly along the vertical dimension (i.e., distance). This study provides evidence of the functional relevance of hV6A in action reprogramming while a sudden event requires a change in performance and shows that hV6A also plays a role in state estimation during reaching. These findings are in line with neurological data showing impairments in actions performed along the distance dimension when lesions occur in the dorsal posterior parietal cortex.

Biases in the spectral amplitude distribution of a natural scene modulate horizontal size perception.

Introduction: Visual perception is a complex process that involves the analysis of different spatial and temporal features of the visual environment. One critical aspect of this process is adaptation, which allows the visual system to adjust its sensitivity to specific features based on the context of the environment. Numerous theories highlight the significance of the visual scene and its spectral properties in perceptual and adaptation mechanisms. For example, size perception is known to be influenced by the spatial frequency content of the visual scene. Nonetheless, several inquiries still exist, including how specific spectral properties of the scene play a role in size perception and adaptation mechanisms. Methods: In this study, we explore aftereffects on size perception following adaptation to a natural scene with a biased spectral amplitude distribution. Twenty participants had to manually estimate the horizontal size of a projected rectangle after adaptation to three visually biased conditions: vertical-biased, non-biased, and horizontal-biased. Size adaptation aftereffects were quantified by comparing the perceptual responses from the non-biased condition with the vertical- and horizontal-biased conditions. Results: We found size perception shifts which were contingent upon the specific orientation and spatial frequency distribution inherent in the amplitude spectra of the adaptation stimuli. Particularly, adaptation to vertical-biased produced a horizontal enlargement, while adaptation to horizontal-biased generated a decrease in the horizontal size perception of the rectangle. On average, size perception was modulated by 5-6%. Discussion: These findings provide supporting evidence for the hypothesis that the neural mechanisms responsible for processing spatial frequency channels are involved in the encoding and perception of size information. The implications for neural mechanisms underlying spatial frequency and size information encoding are discussed.

Functional organization of the caudal part of the human superior parietal lobule.

Like in macaque, the caudal portion of the human superior parietal lobule (SPL) plays a key role in a series of perceptive, visuomotor and somatosensory processes. Here, we review the functional properties of three separate portions of the caudal SPL, i.e., the posterior parieto-occipital sulcus (POs), the anterior POs, and the anterior part of the caudal SPL. We propose that the posterior POs is mainly dedicated to the analysis of visual motion cues useful for object motion detection during self-motion and for spatial navigation, while the more anterior parts are implicated in visuomotor control of limb actions. The anterior POs is mainly involved in using the spotlight of attention to guide reach-to-grasp hand movements, especially in dynamic environments. The anterior part of the caudal SPL plays a central role in visually guided locomotion, being implicated in controlling leg-related movements as well as the four limbs interaction with the environment, and in encoding egomotion-compatible optic flow. Together, these functions reveal how the caudal SPL is strongly implicated in skilled visually-guided behaviors.

Motor decoding from the posterior parietal cortex using deep neural networks.

Objective.Motor decoding is crucial to translate the neural activity for brain-computer interfaces (BCIs) and provides information on how motor states are encoded in the brain. Deep neural networks (DNNs) are emerging as promising neural decoders. Nevertheless, it is still unclear how different DNNs perform in different motor decoding problems and scenarios, and which network could be a good candidate for invasive BCIs.Approach.Fully-connected, convolutional, and recurrent neural networks (FCNNs, CNNs, RNNs) were designed and applied to decode motor states from neurons recorded from V6A area in the posterior parietal cortex (PPC) of macaques. Three motor tasks were considered, involving reaching and reach-to-grasping (the latter under two illumination conditions). DNNs decoded nine reaching endpoints in 3D space or five grip types using a sliding window approach within the trial course. To evaluate decoders simulating a broad variety of scenarios, the performance was also analyzed while artificially reducing the number of recorded neurons and trials, and while performing transfer learning from one task to another. Finally, the accuracy time course was used to analyze V6A motor encoding.Main results.DNNs outperformed a classic Naïve Bayes classifier, and CNNs additionally outperformed XGBoost and Support Vector Machine classifiers across the motor decoding problems. CNNs resulted the top-performing DNNs when using less neurons and trials, and task-to-task transfer learning improved performance especially in the low data regime. Lastly, V6A neurons encoded reaching and reach-to-grasping properties even from action planning, with the encoding of grip properties occurring later, closer to movement execution, and appearing weaker in darkness.Significance.Results suggest that CNNs are effective candidates to realize neural decoders for invasive BCIs in humans from PPC recordings also reducing BCI calibration times (transfer learning), and that a CNN-based data-driven analysis may provide insights about the encoding properties and the functional roles of brain regions.

The influence of action on perception spans different effectors.

Perception and action are fundamental processes that characterize our life and our possibility to modify the world around us. Several pieces of evidence have shown an intimate and reciprocal interaction between perception and action, leading us to believe that these processes rely on a common set of representations. The present review focuses on one particular aspect of this interaction: the influence of action on perception from a motor effector perspective during two phases, action planning and the phase following execution of the action. The movements performed by eyes, hands, and legs have a different impact on object and space perception; studies that use different approaches and paradigms have formed an interesting general picture that demonstrates the existence of an action effect on perception, before as well as after its execution. Although the mechanisms of this effect are still being debated, different studies have demonstrated that most of the time this effect pragmatically shapes and primes perception of relevant features of the object or environment which calls for action; at other times it improves our perception through motor experience and learning. Finally, a future perspective is provided, in which we suggest that these mechanisms can be exploited to increase trust in artificial intelligence systems that are able to interact with humans.

A Short Route for Reach Planning between Human V6A and the Motor Cortex.

In the macaque monkey, area V6A, located in the medial posterior parietal cortex, contains cells that encode the spatial position of a reaching target. It has been suggested that during reach planning this information is sent to the frontal cortex along a parieto-frontal pathway that connects V6A-premotor cortex-M1. A similar parieto-frontal network may also exist in the human brain, and we aimed here to study the timing of this functional connection during planning of a reaching movement toward different spatial positions. We probed the functional connectivity between human area V6A (hV6A) and the primary motor cortex (M1) using dual-site, paired-pulse transcranial magnetic stimulation with a short (4 ms) and a longer (10 ms) interstimulus interval while healthy participants (18 men and 18 women) planned a visually-guided or a memory-guided reaching movement toward positions located at different depths and directions. We found that, when the stimulation over hV6A is sent 4 ms before the stimulation over M1, hV6A inhibits motor-evoked potentials during planning of either rightward or leftward reaching movements. No modulations were found when the stimulation over hV6A was sent 10 ms before the stimulation over M1, suggesting that only short medial parieto-frontal routes are active during reach planning. Moreover, the short route of hV6A-premotor cortex-M1 is active during reach planning irrespectively of the nature (visual or memory) of the reaching target. These results agree with previous neuroimaging studies and provide the first demonstration of the flow of inhibitory signals between hV6A and M1.SIGNIFICANCE STATEMENT All our dexterous movements depend on the correct functioning of the network of brain areas. Knowing the functional timing of these networks is useful to gain a deeper understanding of how the brain works to enable accurate arm movements. In this article, we probed the parieto-frontal network and demonstrated that it takes 4 ms for the medial posterior parietal cortex to send inhibitory signals to the frontal cortex during reach planning. This fast flow of information seems not to be dependent on the availability of visual information regarding the reaching target. This study opens the way for future studies to test how this timing could be impaired in different neurological disorders.

Complementary contribution of the medial and lateral human parietal cortex to grasping: a repetitive TMS study.

The dexterous control of our grasping actions relies on the cooperative activation of many brain areas. In the parietal lobe, 2 grasp-related areas collaborate to orchestrate an accurate grasping action: dorsolateral area AIP and dorsomedial area V6A. Single-cell recordings in monkeys and fMRI studies in humans have suggested that both these areas specify grip aperture and wrist orientation, but encode these grasping parameters differently, depending on the context. To elucidate the causal role of phAIP and hV6A, we stimulated these areas, while participants were performing grasping actions (unperturbed grasping). rTMS over phAIP impaired the wrist orientation process, whereas stimulation over hV6A impaired grip aperture encoding. In a small percentage of trials, an unexpected reprogramming of grip aperture or wrist orientation was required (perturbed grasping). In these cases, rTMS over hV6A or over phAIP impaired reprogramming of both grip aperture and wrist orientation. These results represent the first direct demonstration of a different encoding of grasping parameters by 2 grasp-related parietal areas.

Anterior-posterior gradient in the integrated processing of forelimb movement direction and distance in macaque parietal cortex.

A major issue in modern neuroscience is to understand how cell populations present multiple spatial and motor features during goal-directed movements. The direction and distance (depth) of arm movements often appear to be controlled independently during behavior, but it is unknown whether they share neural resources or not. Using information theory, singular value decomposition, and dimensionality reduction methods, we compare direction and depth effects and their convergence across three parietal areas during an arm movement task. All methods show a stronger direction effect during early movement preparation, whereas depth signals prevail during movement execution. Going from anterior to posterior sectors, we report an increased number of cells processing both signals and stronger depth effects. These findings suggest a serial direction and depth processing consistent with behavioral evidence and reveal a gradient of joint versus independent control of these features in parietal cortex that supports its role in sensorimotor transformations.

New insights on single-neuron selectivity in the era of population-level approaches.

In the past, neuroscience was focused on individual neurons seen as the functional units of the nervous system, but this approach fell short over time to account for new experimental evidence, especially for what concerns associative and motor cortices. For this reason and thanks to great technological advances, a part of modern research has shifted the focus from the responses of single neurons to the activity of neural ensembles, now considered the real functional units of the system. However, on a microscale, individual neurons remain the computational components of these networks, thus the study of population dynamics cannot prescind from studying also individual neurons which represent their natural substrate. In this new framework, ideas such as the capability of single cells to encode a specific stimulus (neural selectivity) may become obsolete and need to be profoundly revised. One step in this direction was made by introducing the concept of "mixed selectivity," the capacity of single cells to integrate multiple variables in a flexible way, allowing individual neurons to participate in different networks. In this review, we outline the most important features of mixed selectivity and we also present recent works demonstrating its presence in the associative areas of the posterior parietal cortex. Finally, in discussing these findings, we present some open questions that could be addressed by future studies.

Machine learning methods detect arm movement impairments in a patient with parieto-occipital lesion using only early kinematic information.

Patients with lesions of the parieto-occipital cortex typically misreach visual targets that they correctly perceive (optic ataxia). Although optic ataxia was described more than 30 years ago, distinguishing this condition from physiological behavior using kinematic data is still far from being an achievement. Here, combining kinematic analysis with machine learning methods, we compared the reaching performance of a patient with bilateral occipitoparietal damage with that of 10 healthy controls. They performed visually guided reaches toward targets located at different depths and directions. Using the horizontal, sagittal, and vertical deviation of the trajectories, we extracted classification accuracy in discriminating the reaching performance of patient from that of controls. Specifically, accurate predictions of the patient's deviations were detected after the 20% of the movement execution in all the spatial positions tested. This classification based on initial trajectory decoding was possible for both directional and depth components of the movement, suggesting the possibility of applying this method to characterize pathological motor behavior in wider frameworks.

The human middle temporal cortex responds to both active leg movements and egomotion-compatible visual motion.

The human middle-temporal region MT+ is highly specialized in processing visual motion. However, recent studies have shown that this region is modulated by extraretinal signals, suggesting a possible involvement in processing motion information also from non-visual modalities. Here, we used functional MRI data to investigate the influence of retinal and extraretinal signals on MT+ in a large sample of subjects. Moreover, we used resting-state functional MRI to assess how the subdivisions of MT+ (i.e., MST, FST, MT, and V4t) are functionally connected. We first compared responses in MST, FST, MT, and V4t to coherent vs. random visual motion. We found that only MST and FST were positively activated by coherent motion. Furthermore, regional analyses revealed that MST and FST were positively activated by leg, but not arm, movements, while MT and V4t were deactivated by arm, but not leg, movements. Taken together, regional analyses revealed a visuomotor role for the anterior areas MST and FST and a pure visual role for the anterior areas MT and V4t. These findings were mirrored by the pattern of functional connections between these areas and the rest of the brain. Visual and visuomotor regions showed distinct patterns of functional connectivity, with the latter preferentially connected with the somatosensory and motor areas representing leg and foot. Overall, these findings reveal a functional sensitivity for coherent visual motion and lower-limb movements in MST and FST, suggesting their possible involvement in integrating sensory and motor information to perform locomotion.

The posterior parietal area V6A: An attentionally-modulated visuomotor region involved in the control of reach-to-grasp action.

In the macaque, the posterior parietal area V6A is involved in the control of all phases of reach-to-grasp actions: the transport phase, given that reaching neurons are sensitive to the direction and amplitude of arm movement, and the grasping phase, since reaching neurons are also sensitive to wrist orientation and hand shaping. Reaching and grasping activity are corollary discharges which, together with the somatosensory and visual signals related to the same movement, allow V6A to act as a state estimator that signals discrepancies during the motor act in order to maintain consistency between the ongoing movement and the desired one. Area V6A is also able to encode the target of an action because of gaze-dependent visual neurons and real-position cells. Here, we advance the hypothesis that V6A also uses the spotlight of attention to guide goal-directed movements of the hand, and hosts a priority map that is specific for the guidance of reaching arm movement, combining bottom-up inputs such as visual responses with top-down signals such as reaching plans.

Egomotion-related visual areas respond to goal-directed movements.

Integration of proprioceptive signals from the various effectors with visual feedback of self-motion from the retina is necessary for whole-body movement and locomotion. Here, we tested whether the human visual motion areas involved in processing optic flow signals simulating self-motion are also activated by goal-directed movements (as saccades or pointing) performed with different effectors (eye, hand, and foot), suggesting a role in visually guiding movements through the external environment. To achieve this aim, we used a combined approach of task-evoked activity and effective connectivity (PsychoPhysiological Interaction, PPI) by fMRI. We localized a set of six egomotion-responsive visual areas through the flow field stimulus and distinguished them into visual (pIPS/V3A, V6+ , IPSmot/VIP) and visuomotor (pCi, CSv, PIC) areas according to recent literature. We tested their response to a visuomotor task implying spatially directed delayed eye, hand, and foot movements. We observed a posterior-to-anterior gradient of preference for eye-to-foot movements, with posterior (visual) regions showing a preference for saccades, and anterior (visuomotor) regions showing a preference for foot pointing. No region showed a clear preference for hand pointing. Effective connectivity analysis showed that visual areas were more connected to each other with respect to the visuomotor areas, particularly during saccades. We suggest that visual and visuomotor egomotion regions can play different roles within a network that integrates sensory-motor signals with the aim of guiding movements in the external environment.

Decoding sensorimotor information from superior parietal lobule of macaque via Convolutional Neural Networks.

Despite the well-recognized role of the posterior parietal cortex (PPC) in processing sensory information to guide action, the differential encoding properties of this dynamic processing, as operated by different PPC brain areas, are scarcely known. Within the monkey's PPC, the superior parietal lobule hosts areas V6A, PEc, and PE included in the dorso-medial visual stream that is specialized in planning and guiding reaching movements. Here, a Convolutional Neural Network (CNN) approach is used to investigate how the information is processed in these areas. We trained two macaque monkeys to perform a delayed reaching task towards 9 positions (distributed on 3 different depth and direction levels) in the 3D peripersonal space. The activity of single cells was recorded from V6A, PEc, PE and fed to convolutional neural networks that were designed and trained to exploit the temporal structure of neuronal activation patterns, to decode the target positions reached by the monkey. Bayesian Optimization was used to define the main CNN hyper-parameters. In addition to discrete positions in space, we used the same network architecture to decode plausible reaching trajectories. We found that data from the most caudal V6A and PEc areas outperformed PE area in the spatial position decoding. In all areas, decoding accuracies started to increase at the time the target to reach was instructed to the monkey, and reached a plateau at movement onset. The results support a dynamic encoding of the different phases and properties of the reaching movement differentially distributed over a network of interconnected areas. This study highlights the usefulness of neurons' firing rate decoding via CNNs to improve our understanding of how sensorimotor information is encoded in PPC to perform reaching movements. The obtained results may have implications in the perspective of novel neuroprosthetic devices based on the decoding of these rich signals for faithfully carrying out patient's intentions.

A Multifunctional Adaptive and Interactive AI system to support people living with stroke, acquired brain or spinal cord injuries: A study protocol.

BACKGROUND: Acquired brain injury and spinal cord injury are leading causes of severe motor disabilities impacting a person's autonomy and social life. Enhancing neurological recovery driven by neurogenesis and neuronal plasticity could represent future solutions; however, at present, recovery of activities employing assistive technologies integrating artificial intelligence is worthy of examining. MAIA (Multifunctional, adaptive, and interactive AI system for Acting in multiple contexts) is a human-centered AI aiming to allow end-users to control assistive devices naturally and efficiently by using continuous bidirectional exchanges among multiple sensorimotor information. METHODS: Aimed at exploring the acceptability of MAIA, semi-structured interviews (both individual interviews and focus groups) are used to prompt possible end-users (both patients and caregivers) to express their opinions about expected functionalities, outfits, and the services that MAIA should embed, once developed, to fit end-users needs. DISCUSSION: End-user indications are expected to interest MAIA technical, health-related, and setting components. Moreover, psycho-social issues are expected to align with the technology acceptance model. In particular, they are likely to involve intrinsic motivational and extrinsic social aspects, aspects concerning the usefulness of the MAIA system, and the related ease to use. At last, we expect individual factors to impact MAIA: gender, fragility levels, psychological aspects involved in the mental representation of body image, personal endurance, and tolerance toward AT-related burden might be the aspects end-users rise in evaluating the MAIA project.

Horizontal target size perturbations during grasping movements are described by subsequent size perception and saccade amplitude.

Perception and action are essential in our day-to-day interactions with the environment. Despite the dual-stream theory of action and perception, it is now accepted that action and perception processes interact with each other. However, little is known about the impact of unpredicted changes of target size during grasping actions on perception. We assessed whether size perception and saccade amplitude were affected before and after grasping a target that changed its horizontal size during the action execution under the presence or absence of tactile feedback. We have tested twenty-one participants in 4 blocks of 30 trials. Blocks were divided into two experimental tactile feedback paradigms: tactile and non-tactile. Trials consisted of 3 sequential phases: pre-grasping size perception, grasping, and post-grasping size perception. During pre- and post-phases, participants executed a saccade towards a horizontal bar and performed a manual size estimation of the bar size. During grasping phase, participants were asked to execute a saccade towards the bar and to make a grasping action towards the screen. While grasping, 3 horizontal size perturbation conditions were applied: non-perturbation, shortening, and lengthening. 30% of the trials presented perturbation, meaning a symmetrically shortened or lengthened by 33% of the original size. Participants' hand and eye positions were assessed by a motion capture system and a mobile eye-tracker, respectively. After grasping, in both tactile and non-tactile feedback paradigms, size estimation was significantly reduced in lengthening (p = 0.002) and non-perturbation (p<0.001), whereas shortening did not induce significant adjustments (p = 0.86). After grasping, saccade amplitude became significantly longer in shortening (p<0.001) and significantly shorter in lengthening (p<0.001). Non-perturbation condition did not display adjustments (p = 0.95). Tactile feedback did not generate changes in the collected perceptual responses, but horizontal size perturbations did so, suggesting that all relevant target information used in the movement can be extracted from the post-action target perception.