Computation on demand: Action-specific representations of visual task features arise during distinct movement phases.

The intention to act influences the computations of various task-relevant features. However, little is known about the time course of these computations. Furthermore, it is commonly held that these computations are governed by conjunctive neural representations of the features. But, support for this view comes from paradigms arbitrarily combining task features and affordances, thus, requiring representations in working memory. Therefore, the present study used electroencephalography and a well-rehearsed task with features that afford minimal working memory representations to investigate the temporal evolution of feature representations and their potential integration in the brain. Female and male human participants viewed and grasped objects or touched them with a knuckle. Objects had different shapes and were made of heavy or light materials with shape and weight being relevant for grasping, not for "knuckling." Using multivariate analysis showed that representations of object shape were similar for grasping and knuckling. However, only for grasping did early shape representations reactivate at later phases of grasp planning, suggesting that sensorimotor control signals feed back to early visual cortex. Grasp-specific representations of material/weight only arose during grasp execution after object contact during the load phase. A trend for integrated representations of shape and material also became grasp-specific but only briefly during Movement onset. These results suggest that the brain generates action-specific representations of relevant features as required for the different subcomponents of its action computations. Our results argue against the view that goal-directed actions inevitably join all features of a task into a sustained and unified neural representation.Significance statement The idea that all the features of a task are integrated into a joint representation or event file is widely supported but importantly based on paradigms with arbitrary stimulus-response combinations. Our study is the first to investigate grasping using electroencephalography to search for the neural basis of feature integration in such a daily-life task with overlearned stimulus-response mappings. Contrary to the notion of event files we find limited evidence for integrated representations. Instead, we find that task-relevant features form representations at specific phases of the action, suggesting that action intentions reactivate representations of relevant features. Our results show that integrated representations do not occur universally for any kind of goal-directed behaviour but in a manner of computation on demand.

Working memory in action: Transsaccadic working memory deficits in the left visual field and after transcallosal remapping.

Every waking second, we make three saccadic eye movements that move our retinal images. Thus, to attain a coherent image of the world we need to remember visuo-spatial information across saccades. But transsaccadic working memory (tWM) remains poorly understood. Crucially, there has been a debate whether there are any differences in tWM for the left vs. right visual field and depending on saccade direction. However, previous studies have probed tWM with minimal loads whereas spatial differences might arise with higher loads. Here we employed a task that probed higher memory load for spatial information in the left and right visual field and with horizontal as well as vertical saccades. We captured several measures of precision and accuracy of performance that, when submitted to principal component analysis, produced two components. Component 1, mainly associated with precision, yielded greater error for the left than the right visual field. Component 2 was associated with performance accuracy and unexpectedly produced a disadvantage after rightward saccades. Both components showed that performance was worse when rightward or leftward saccades afforded a shift of memory representations between visual fields compared to remapping within the same field. Our study offers several novel findings. It is the first to show that tWM involves at least two components likely reflecting working memory capacity and strategic aspects of working memory, respectively. Reduced capacity for the left, rather than the right visual field is consistent with how the left and right visual fields are known to be represented in the two hemispheres. Remapping difficulties between visual fields is consistent with the limited information transfer across the corpus callosum. Finally, the impact of rightward saccades on working memory might be due to greater interference of the accompanying shifts of attention. Our results highlight the dynamic nature of transsaccadic working memory.

Saccades and presaccadic stimulus repetition alter cortical network topology and dynamics: evidence from EEG and graph theoretical analysis.

Parietal and frontal cortex are involved in saccade generation, and their output signals modify visual signals throughout cortex. Local signals associated with these interactions are well described, but their large-scale progression and network dynamics are unknown. Here, we combined source localized electroencephalography (EEG) and graph theory analysis (GTA) to understand how saccades and presaccadic visual stimuli interactively alter cortical network dynamics in humans. Twenty-one participants viewed 1-3 vertical/horizontal grids, followed by grid with the opposite orientation just before a horizontal saccade or continued fixation. EEG signals from the presaccadic interval (or equivalent fixation period) were used for analysis. Source localization-through-time revealed a rapid frontoparietal progression of presaccadic motor signals and stimulus-motor interactions, with additional band-specific modulations in several frontoparietal regions. GTA analysis revealed a saccade-specific functional network with major hubs in inferior parietal cortex (alpha) and the frontal eye fields (beta), and major saccade-repetition interactions in left prefrontal (theta) and supramarginal gyrus (gamma). This network showed enhanced segregation, integration, synchronization, and complexity (compared with fixation), whereas stimulus repetition interactions reduced synchronization and complexity. These cortical results demonstrate a widespread influence of saccades on both regional and network dynamics, likely responsible for both the motor and perceptual aspects of saccades.

Do motor plans affect sensorimotor state estimates during temporal decision-making with crossed vs. uncrossed hands? Failure to replicate the dynamic crossed-hand effect.

Hermosillo et al. (J Neurosci 31: 10019-10022, 2011) have suggested that action planning of hand movements impacts decisions about the temporal order judgments regarding vibrotactile stimulation of the hands. Specifically, these authors reported that the crossed-hand effect, a confusion about which hand is which when held in a crossed posture, gradually reverses some 320 ms before the arms begin to move from an uncrossed to a crossed posture or vice versa, such that the crossed-hand is reversed at the time of movement onset in anticipation of the movement's end position. However, to date, no other study has attempted to replicate this dynamic crossed-hand effect. Therefore, in the present study, we conducted four experiments to revisit the question whether preparing uncrossed-to-crossed or crossed-to-uncrossed movements affects the temporo-spatial perception of tactile stimulation of the hands. We used a temporal order judgement (TOJ) task at different time stages during action planning to test whether TOJs are more difficult with crossed than uncrossed hands ("static crossed-hand effect") and, crucially, whether planning to cross or uncross the hands shows the opposite pattern of difficulties ("dynamic crossed-hand effect"). As expected, our results confirmed the static crossed-hand effect. However, the dynamic crossed-hand effect could not be replicated. In addition, we observed that participants delayed their movements with late somatosensory stimulation from the TOJ task, even when the stimulations were meaningless, suggesting that the TOJ task resulted in cross-modal distractions. Whereas the current findings are not inconsistent with a contribution of motor signals to posture perception, they cast doubt on observations that motor signals impact state estimates well before movement onset.

Linear vector models of time perception account for saccade and stimulus novelty interactions.

Various models (e.g., scalar, state-dependent network, and vector models) have been proposed to explain the global aspects of time perception, but they have not been tested against specific visual phenomena like perisaccadic time compression and novel stimulus time dilation. Here, in two separate experiments (N = 31), we tested how the perceived duration of a novel stimulus is influenced by 1) a simultaneous saccade, in combination with 2) a prior series of repeated stimuli in human participants. This yielded a novel behavioral interaction: pre-saccadic stimulus repetition neutralizes perisaccadic time compression. We then tested these results against simulations of the above models. Our data yielded low correlations against scalar model simulations, high but non-specific correlations for our feedforward neural network, and correlations that were both high and specific for a vector model based on identity of objective and subjective time. These results demonstrate the power of global time perception models in explaining disparate empirical phenomena and suggest that subjective time has a similar essence to time's physical vector.

Multivariate Analysis of Electrophysiological Signals Reveals the Time Course of Precision Grasps Programs: Evidence for Nonhierarchical Evolution of Grasp Control.

Current understanding of the neural processes underlying human grasping suggests that grasp computations involve gradients of higher to lower level representations and, relatedly, visual to motor processes. However, it is unclear whether these processes evolve in a strictly canonical manner from higher to intermediate and to lower levels given that this knowledge importantly relies on functional imaging, which lacks temporal resolution. To examine grasping in fine temporal detail here we used multivariate EEG analysis. We asked participants to grasp objects while controlling the time at which crucial elements of grasp programs were specified. We first specified the orientation with which participants should grasp objects, and only after a delay we instructed participants about which effector to use to grasp, either the right or the left hand. We also asked participants to grasp with both hands because bimanual and left-hand grasping share intermediate-level grasp representations. We observed that grasp programs evolved in a canonical manner from visual representations, which were independent of effectors to motor representations that distinguished between effectors. However, we found that intermediate representations of effectors that partially distinguished between effectors arose after representations that distinguished among all effector types. Our results show that grasp computations do not proceed in a strictly hierarchically canonical fashion, highlighting the importance of the fine temporal resolution of EEG for a comprehensive understanding of human grasp control.SIGNIFICANCE STATEMENT A long-standing assumption of the grasp computations is that grasp representations progress from higher to lower level control in a regular, or canonical, fashion. Here, we combined EEG and multivariate pattern analysis to characterize the temporal dynamics of grasp representations while participants viewed objects and were subsequently cued to execute an unimanual or bimanual grasp. Interrogation of the temporal dynamics revealed that lower level effector representations emerged before intermediate levels of grasp representations, thereby suggesting a partially noncanonical progression from higher to lower and then to intermediate level grasp control.

Is the n-back task a measure of unstructured working memory capacity? Towards understanding its connection to other working memory tasks.

Working memory is fundamental to human cognitive functioning, and it is often measured with the n-back task. However, it is not clear whether the n-back task is a valid measure of working memory. Importantly, previous studies have found poor correlations with measures of complex span, whereas a recent study (Frost et al., 2019) showed that n-back performance was correlated with a transsaccadic memory task but dissociated from performance on the change detection task, a well-accepted measure of working memory capacity. To test whether capacity is involved in the n-back task we correlated a spatial version of the test with different versions of the change detection task. Experiment 1 introduced perceptual and cognitive disruptions to the change detection task. This impacted task performance, however, all versions of the change detection task remained highly correlated with one another whereas there was no significant correlation with the n-back task. Experiment 2 removed spatial and non-spatial context from the change detection task. This produced a correlation with n-back. Our results indicate that the n-back task is supported by faculties similar to those that support change detection, but that this commonality is hidden when contextual information is available to be exploited in a change detection task such that structured representations can form. We suggest that n-back might be a valid measure of working memory, and that the ability to exploit contextual information is an important faculty captured by some versions of the change detection task.

Grasping of Real-World Objects Is Not Biased by Ensemble Perception.

The visual system is known to extract summary representations of visually similar objects which bias the perception of individual objects toward the ensemble average. Although vision plays a large role in guiding action, less is known about whether ensemble representation is informative for action. Motor behavior is tuned to the veridical dimensions of objects and generally considered resistant to perceptual biases. However, when the relevant grasp dimension is not available or is unconstrained, ensemble perception may be informative to behavior by providing gist information about surrounding objects. In the present study, we examined if summary representations of a surrounding ensemble display influenced grip aperture and orientation when participants reached-to-grasp a central circular target which had an explicit size but importantly no explicit orientation that the visuomotor system could selectively attend to. Maximum grip aperture and grip orientation were not biased by ensemble statistics during grasping, although participants were able to perceive and provide manual estimations of the average size and orientation of the ensemble display. Support vector machine classification of ensemble statistics achieved above-chance classification accuracy when trained on kinematic and electromyography data of the perceptual but not grasping conditions, supporting our univariate findings. These results suggest that even along unconstrained grasping dimensions, visually-guided behaviors toward real-world objects are not biased by ensemble processing.

Critical incident reporting systems (CIRS) in trauma patients may identify common quality problems.

BACKGROUND: Critical incident reporting systems (CIRS) are considered to be a valid instrument to identify typical errors in various clinical settings as well as in prehospital emergency medicine. Our aim was to review incidents and errors in the care of trauma patients during the period of emergency trauma room treatment before their transfer to the intensive care unit or the operation room. METHODS: We screened six open access and German language-based CIRS-platforms on the internet. RESULTS: We identified 78 critical incidents. They could be divided into four groups: organization related (n = 30), communication related (n = 6), equipment related (n = 28), and medical error (n = 23). Within the category, typical, common, or frequent clusters were identified, such as incomplete trauma team, malfunctioning equipment, or a lack of communication skills. In 12 cases (15.4%), patients were reported to have been harmed, mostly by medical errors. Three reported incidents (3.6%) were considered near-incidents. CONCLUSIONS: Our results demonstrate that using CIRS is able to reveal individual or rare errors and allows for the identification of systematic errors and deficiencies in the acute care of trauma patients in the trauma room. This may guide quality control and quality improvement measures to be focused on the most common fields of demand.

Mapping the anatomy of perceptual pseudoneglect. A multivariate approach.

Fundamental to the understanding of the functions of spatial cognition and attention is to clarify the underlying neural mechanisms. It is clear that relatively right-dominant activity in ventral and dorsal parieto-frontal cortex is associated with attentional reorienting, certain forms of mental imagery and spatial working memory for higher loads, while lesions mostly to right ventral areas cause spatial neglect with pathological attentional biases to the right side. In contrast, complementary leftward biases in healthy people, called pseudoneglect, have been associated with varying patterns of cortical activity. Notably, this inconsistency may be explained, at least in part, by the fact that pseudoneglect studies have often employed experimental paradigms that do not control sufficiently for cognitive processes unrelated to pseudoneglect. To address this issue, here we administered a carefully designed continuum of pseudoneglect and control tasks in healthy adults while using functional magnetic resonance imaging (fMRI). Data submitted to partial least square (PLS) imaging analysis yielded a significant latent variable that identified a right-dominant network of brain regions along the intra-occipital and -parietal sulci, frontal eye fields and right ventral cortex in association with perceptual pseudoneglect. Our results shed new light on the interplay of attentional and cognitive systems in pseudoneglect.

Multivariate Analysis of Electrophysiological Signals Reveals the Temporal Properties of Visuomotor Computations for Precision Grips.

The frontoparietal networks underlying grasping movements have been extensively studied, especially using fMRI. Accordingly, whereas much is known about their cortical locus much less is known about the temporal dynamics of visuomotor transformations. Here, we show that multivariate EEG analysis allows for detailed insights into the time course of visual and visuomotor computations of precision grasps. Male and female human participants first previewed one of several objects and, upon its reappearance, reached to grasp it with the thumb and index finger along one of its two symmetry axes. Object shape classifiers reached transient accuracies of 70% at ∼105 ms, especially based on scalp sites over visual cortex, dropping to lower levels thereafter. Grasp orientation classifiers relied on a system of occipital-to-frontal electrodes. Their accuracy rose concurrently with shape classification but ramped up more gradually, and the slope of the classification curve predicted individual reaction times. Further, cross-temporal generalization revealed that dynamic shape representation involved early and late neural generators that reactivated one another. In contrast, grasp computations involved a chain of generators attaining a sustained state about 100 ms before movement onset. Our results reveal the progression of visual and visuomotor representations over the course of planning and executing grasp movements.SIGNIFICANCE STATEMENT Grasping an object requires the brain to perform visual-to-motor transformations of the object's properties. Although much of the neuroanatomic basis of visuomotor transformations has been uncovered, little is known about its time course. Here, we orthogonally manipulated object visual characteristics and grasp orientation, and used multivariate EEG analysis to reveal that visual and visuomotor computations follow similar time courses but display different properties and dynamics.

Working memory in action: inspecting the systematic and unsystematic errors of spatial memory across saccades.

Our ability to interact with the world depends on memory buffers that flexibly store and process information for short periods of time. Current working memory research, however, mainly uses tasks that avoid eye movements, whereas in daily life we need to remember information across saccades. Because saccades disrupt perception and attention, the brain might use special transsaccadic memory systems. Therefore, to compare working memory systems between and across saccades, the current study devised transsaccadic memory tasks that evaluated the influence of memory load on several kinds of systematic and unsystematic spatial errors, and tested whether these measures predicted performance in more established working memory paradigms. Experiment 1 used a line intersection task that had people integrate lines shown before and after saccades, and it administered a 2-back task. Experiments 2 and 3 asked people to point at one of several locations within a memory array flashed before an eye movement, and we tested change detection and 2-back performance. We found that unsystematic transsaccadic errors increased with memory load and were correlated with 2-back performance. Systematic errors produced similar results, although effects varied as a function of the geometric layout of the memory arrays. Surprisingly, transsaccadic errors did not predict change detection performance despite the latter being a widely accepted measure of working memory capacity. Our results suggest that working memory systems between and across saccades share, in part, similar neural resources. Nevertheless, our data highlight the importance of investigating working memory across saccades.

Evidence for a common mechanism of spatial attention and visual awareness: Towards construct validity of pseudoneglect.

Present knowledge of attention and awareness centres on deficits in patients with right brain damage who show severe forms of inattention to the left, called spatial neglect. Yet the functions that are lost in neglect are poorly understood. In healthy people, they might produce "pseudoneglect"-subtle biases to the left found in various tests that could complement the leftward deficits in neglect. But pseudoneglect measures are poorly correlated. Thus, it is unclear whether they reflect anything but distinct surface features of the tests. To probe for a common mechanism, here we asked whether visual noise, known to increase leftward biases in the grating-scales task, has comparable effects on other measures of pseudoneglect. We measured biases using three perceptual tasks that require judgments about size (landmark task), luminance (greyscales task) and spatial frequency (grating-scales task), as well as two visual search tasks that permitted serial and parallel search or parallel search alone. In each task, we randomly selected pixels of the stimuli and set them to random luminance values, much like a poor TV signal. We found that participants biased their perceptual judgments more to the left with increasing levels of noise, regardless of task. Also, noise amplified the difference between long and short lines in the landmark task. In contrast, biases during visual searches were not influenced by noise. Our data provide crucial evidence that different measures of perceptual pseudoneglect, but not exploratory pseudoneglect, share a common mechanism. It can be speculated that this common mechanism feeds into specific, right-dominant processes of global awareness involved in the integration of visual information across the two hemispheres.

Emergent Synergistic Grasp-Like Behavior in a Visuomotor Joint Action Task: Evidence for Internal Forward Models as Building Blocks of Human Interactions.

Central to the mechanistic understanding of the human mind is to clarify how cognitive functions arise from simpler sensory and motor functions. A longstanding assumption is that forward models used by sensorimotor control to anticipate actions also serve to incorporate other people's actions and intentions, and give rise to sensorimotor interactions between people, and even abstract forms of interactions. That is, forward models could aid core aspects of human social cognition. To test whether forward models can be used to coordinate interactions, here we measured the movements of pairs of participants in a novel joint action task. For the task they collaborated to lift an object, each of them using fingers of one hand to push against the object from opposite sides, just like a single person would use two hands to grasp the object bimanually. Perturbations of the object were applied randomly as they are known to impact grasp-specific movement components in common grasping tasks. We found that co-actors quickly learned to make grasp-like movements with grasp components that showed coordination on average based on action observation of peak deviation and velocity of their partner's trajectories. Our data suggest that co-actors adopted pre-existing bimanual grasp programs for their own body to use forward models of their partner's effectors. This is consistent with the long-held assumption that human higher-order cognitive functions may take advantage of sensorimotor forward models to plan social behavior. New and Noteworthy: Taking an approach of sensorimotor neuroscience, our work provides evidence for a long-held belief that the coordination of physical as well as abstract interactions between people originates from certain sensorimotor control processes that form mental representations of people's bodies and actions, called forward models. With a new joint action paradigm and several new analysis approaches we show that, indeed, people coordinate each other's interactions based on forward models and mutual action observation.

Shared right-hemispheric representations of sensorimotor goals in dynamic task environments.

Functional behaviour affords that we form goals to integrate sensory information about the world around us with suitable motor actions, such as when we plan to grab an object with a hand. However, much research has tested grasping in static scenarios where goals are pursued with repetitive movements, whereas dynamic contexts require goals to be pursued even when changes in the environment require a change in the actions to attain them. To study grasp goals in dynamic environments here, we employed a task where the goal remained the same but the execution of the movement changed; we primed participants to grasp objects either with their right or left hand, and occasionally they had to switch to grasping with both. Switch costs should be minimal if grasp goal representations were used continuously, for example, within the left dominant hemisphere. But remapped or re-computed goal representations should delay movements. We found that switching from right-hand grasping to bimanual grasping delayed reaction times but switching from left-hand grasping to bimanual grasping did not. Further, control experiments showed that the lateralized switch costs were not caused by asymmetric inhibition between hemispheres or switches between usual and unusual tasks. Our results show that the left hemisphere does not serve a general role of sensorimotor grasp goal representation. Instead, sensorimotor grasp goals appear to be represented at intermediate levels of abstraction, downstream from cognitive task representations, yet upstream from the control of the grasping effectors.

Multimodal evidence on shape and surface information in individual face processing.

The significance of shape and surface information for face perception is well established, yet their relative contribution to recognition and their neural underpinnings await clarification. Here, we employ image reconstruction to retrieve, assess and visualize such information using behavioral, electroencephalography and functional magnetic resonance imaging data. Our results indicate that both shape and surface information can be successfully recovered from each modality but that the latter is better recovered than the former, consistent with its key role for face representations. Further, shape and surface information exhibit similar spatiotemporal profiles, rely on the extraction of specific visual features, such as eye shape or skin tone, and reveal a systematic representational structure, albeit with more cross-modal consistency for shape than surface. More generally, the present work illustrates a novel approach to relating and comparing different modalities in terms of perceptual information content. Thus, our results help elucidate the representational basis of individual face recognition while, methodologically, they showcase the utility of image reconstruction and clarify its reliance on diagnostic visual information.

The Neural Dynamics of Facial Identity Processing: Insights from EEG-Based Pattern Analysis and Image Reconstruction.

Uncovering the neural dynamics of facial identity processing along with its representational basis outlines a major endeavor in the study of visual processing. To this end, here, we record human electroencephalography (EEG) data associated with viewing face stimuli; then, we exploit spatiotemporal EEG information to determine the neural correlates of facial identity representations and to reconstruct the appearance of the corresponding stimuli. Our findings indicate that multiple temporal intervals support: facial identity classification, face space estimation, visual feature extraction and image reconstruction. In particular, we note that both classification and reconstruction accuracy peak in the proximity of the N170 component. Further, aggregate data from a larger interval (50-650 ms after stimulus onset) support robust reconstruction results, consistent with the availability of distinct visual information over time. Thus, theoretically, our findings shed light on the time course of face processing while, methodologically, they demonstrate the feasibility of EEG-based image reconstruction.

Altered perceptual pseudoneglect in ADHD: Evidence for a functional disconnection from early visual activation.

Novel insights into the right-brain dominant functions of spatial attention and visual awareness may come from the peculiar observation that the attentional bias to the left in healthy individuals, called "pseudoneglect," increases with visual noise superimposed onto test stimuli. However, it is unclear if this effect originates from noise activating early visual areas or causing higher-level cognitive interference. Cognitive distraction and load are known to induce neglect-like rightward biases in attention deficit hyperactivity disorder (ADHD). Therefore, here we tested pseudoneglect in 21 adults with ADHD using a grating-scales task (GST) in a high (HI) and a low (LO) spatial-frequency condition with superimposed pixel noise. As expected, we found that healthy participants (n =32) displayed a "cross-over" of HI vs. LO biases that increased significantly with noise. However, the ADHD group exhibited no pseudoneglect or cross-over, and noise caused neither rightward nor leftward biases. Furthermore, ADHD individuals produced psychometric functions with normal slopes, indicating normal perceptual sensitivity. Our results show that pseudoneglect is altered in ADHD, but that pixel noise induces no neglect-like rightward biases as this would be expected if pixel noise caused cognitive interference. This suggests that pixel noise has a bottom-up perceptual effect on pseudoneglect. What is more, individuals with ADHD seem to lack activation of attentional functions via sensory stimulation despite intact visual processes. Our study adds to the growing literature of right hemisphere pathology in ADHD and the understanding of sensory noise as an activating factor of visuospatial attention and awareness.