Space, time and number: common coding mechanisms and interactions between domains.

Space, time and number are key dimensions that underlie how we perceive, identify and act within the environment. They are interconnected in our behaviour and brain. In this study, we examined interdependencies between these dimensions. To this end, left- and right-handed participants performed an object collision task that required space-time processing and arithmetic tests that involved number processing. Handedness of the participants influenced collision detection with left-handers being more accurate than right-handers, which is in line with the premise that hand preference guides individual differences as a result of sensorimotor experiences and distinct interhemispheric integration patterns. The data further showed that successful collision detection was a predictor for arithmetic achievement, at least in right-handers. These findings suggest that handedness plays a mediating role in binding information processing across domains, likely due to selective connectivity properties within the sensorimotor system that is guided by hemispheric lateralisation patterns.

Manual dexterity: Functional lateralisation patterns and motor efficiency.

Manual tasks are an important goal-directed ability. In this EEG work, we studied how handedness affects the hemispheric lateralisation patterns during performance of visually-driven movements with either hand. The neural correlates were assessed by means of EEG coherence whereas behavioural output was measured by motor error. The EEG data indicated that left- and right-handers showed distinct recruitment patterns. These involved local interactions between brain regions as well as more widespread associations between brain systems. Despite these differences, brain-behaviour correlations highlighted that motor efficiency depended on left-sided brain regions across groups. These results suggest that skilled hand motor control relies on different neural patterns as a function of handedness whereas behavioural efficiency is linked with the left hemisphere. In conclusion, the present findings add to our understanding about principles of lateralised organisation as a function of handedness.

Cross-Subject EEG Feedback for Implicit Image Generation.

Generative models are powerful tools for producing novel information by learning from example data. However, the current approaches require explicit manual input to steer generative models to match human goals. Furthermore, how these models would integrate implicit, diverse feedback and goals of multiple users remains largely unexplored. Here, we present a first-of-its-kind system that produces novel images of faces by inferring human goals directly from cross-subject brain signals while study subjects are looking at example images. We report on an experiment where brain responses to images of faces were recorded using electroencephalography in 30 subjects, focusing on specific salient visual features (VFs). Preferences toward VFs were decoded from subjects' brain responses and used as implicit feedback for a generative adversarial network (GAN), which generated new images of faces. The results from a follow-up user study evaluating the presence of the target salient VFs show that the images generated from brain feedback represent the goal of the study subjects and are comparable to images generated with manual feedback. The methodology provides a stepping stone toward humans-in-the-loop image generation.

Cognitive control of response inhibition and switching: hemispheric lateralization and hand preference.

Changing environmental constraints often make already prepared responses unnecessary or inappropriate. Under such circumstances, cognitive control enables to suppress the response or switch to alternative behavior. Here, we examine the neural dynamics of both functions in left- and right-handers who performed two variants of a paradigm that required either inhibition of the response or switching the response between hands. The EEG coherence data showed strengthening of interregional coupling in the alpha band (8-12 Hz) following the target cues with an essential contribution of fronto-medial circuitry and a specific involvement of parietal areas in response switching. Brain-behavioral correlations revealed the functional significance of left-sided regions for successful response inhibition and switching, underlining the significant role of the left hemisphere for the organization of goal-directed activities. This lateralization pattern was observed for both left- and right-handers and suggests dominance of higher-order aspects of action planning in the left hemisphere.

Time to imagine moving: Simulated motor activity affects time perception.

Sensing the passage of time is important for countless daily tasks, yet time perception is easily influenced by perception, cognition, and emotion. Mechanistic accounts of time perception have traditionally regarded time perception as part of central cognition. Since proprioception, action execution, and sensorimotor contingencies also affect time perception, perception-action integration theories suggest motor processes are central to the experience of the passage of time. We investigated whether sensory information and motor activity may interactively affect the perception of the passage of time. Two prospective timing tasks involved timing a visual stimulus display conveying optical flow at increasing or decreasing velocity. While doing the timing tasks, participants were instructed to imagine themselves moving at increasing or decreasing speed, independently of the optical flow. In the direct-estimation task, the duration of the visual display was explicitly judged in seconds while in the motor-timing task, participants were asked to keep a constant pace of tapping. The direct-estimation task showed imagining accelerating movement resulted in relative overestimation of time, or time dilation, while decelerating movement elicited relative underestimation, or time compression. In the motor-timing task, imagined accelerating movement also accelerated tapping speed, replicating the time-dilation effect. The experiments show imagined movement affects time perception, suggesting a causal role of simulated motor activity. We argue that imagined movements and optical flow are integrated by temporal unfolding of sensorimotor contingencies. Consequently, as physical time is relative to spatial motion, so too is perception of time relative to imaginary motion.

Actions travel with their objects: evidence for dynamic event files.

Moving a visual object is known to lead to an update of its cognitive representation. Given that object representations have also been shown to include codes describing the actions they were accompanied by, we investigated whether these action codes "move" along with their object. We replicated earlier findings that repeating stimulus and action features enhances performance if other features are repeated, but attenuates performance if they alternate. However, moving the objects in which the stimuli appeared in between two stimulus presentations had a strong impact on the feature bindings that involved location. Taken together, our findings provide evidence that changing the location of an object leaves two memory traces, one referring to its original location (an episodic record) and another referring to the new location (a working-memory trace).

I'll make you an offer you can refuse: How socially sensitive is the MFN?

Medial frontal negativity (MFN) is an event-related potential thought to originate in the anterior cingulate cortex. It is evoked by outcomes being worse than expected, such as when presented with unfair economic proposals during the Ultimatum Game (UG). This could mean the MFN indexes a social-emotional response, as commonly suggested in accounts that relate it to a violation of a social norm of fairness. To examine the link between MFN and norm violation, we designed an EEG experiment with participants acting as representatives in an UG. Participants responded either as themselves, or as representatives of two charities. Of these, a norm-compatible charity conformed to the participant's values, while the norm-incompatible charity contrasted to them. The behavioral results showed that norm-incompatible representation reversed behavior, with almost all fair offers being declined. The MFN, however, was unaffected by the norm representation, with unfair offers consistently evoking MFNs across conditions. We furthermore replicated the curious finding that unexpectedly generous offers evoke as much MFN as unfair offers. Thus, the MFN is not nearly as sensitive to higher-order social-emotional processes as commonly assumed. Instead, the perceived inequality that drives the MFN likely reflects a rational, probabilistic process.

The role of hand dominance and sensorimotor congruence in voluntary movement.

The present study evaluated the neural changes due to effector use (unimanual left, unimanual right, and bimanual) and visuomotor conflict induced by mirror-reversed vision during drawing behavior. EEG phase synchronization, expressing interregional communication, showed that visuomotor incongruence perturbed information processing in both hemispheres. Furthermore, it was observed that the left hemisphere became temporally dominant when movements were executed with visuomotor conflict, independent of the performing hand(s). This observation emphasizes the superiority of the left hemisphere to control complex movements. In addition, the functional interactions between the hemispheres were also perturbed due to visuomotor discordance, indicating the crucial role of interhemispheric communication for movement control. These results highlight that functional connectivity patterns provide higher-order coding mechanisms of information processing. The data further underline the significance of the left hemisphere for intricate visuomotor skills.

Blinks of the eye predict blinks of the mind.

The Attentional Blink (AB)--a deficit in reporting the second of two target stimuli presented in close succession in a rapid sequence of distracters--has been related to individual processing limitations of working memory. Given the known role of dopamine (DA) in working memory processes, the present experiment tested the hypothesis that DA, and in particular the DA/D1 subsystem, plays a role in the AB. We present evidence that the spontaneous eyeblink rate (EBR), a functional marker of central dopaminergic function, reliably predicts the size of AB. Thus, in line with our hypothesis, these data point to a modulatory role for DA in the AB.

He said, she said: episodic retrieval induces conflict adaptation in an auditory Stroop task.

People respond more slowly if an irrelevant feature of a target stimulus is incompatible with the relevant feature or the correct response. Such compatibility effects are often reduced in trials following an incompatible trial, which has been taken to reflect increased cognitive control. This pattern holds only if two trials share some similarities, however, suggesting that it may be modulated by the episodic context. To look into this possibility, we had participants respond to high- or low-pitched tones by saying "high" or "low," respectively, and ignore the simultaneously presented auditory word "high" or "low." As expected, performance was impaired if the heard word was incompatible with the required response, and this Stroop-like effect was reduced after incompatible trials. This sequential modulation was observed, however, only if the voice in the two successive trials was the same, whereas no modulation was obtained when the speaker changed. The results suggest that sequential modulations are due to the automatic retrieval of episodic event representations that integrate stimuli, actions, and situational and task-specific control information, so that later reactivation of some elements of a given representation tends to retrieve the other elements as well.

Motor Timing and Covariation with Time Perception: Investigating the Role of Handedness.

Time is a fundamental dimension of our behavior and enables us to guide our actions and to experience time such as predicting collisions or listening to music. In this study, we investigate the regulation and covariation of motor timing and time perception functions in left- and right-handers who are characterized by distinct brain processing mechanisms for cognitive-motor control. To this purpose, we use a combination of tasks that assess the timed responses during movements and the perception of time intervals. The results showed a positive association across left- and right-handers between movement-driven timing and perceived interval duration when adopting a preferred tempo, suggesting cross-domain coupling between both abilities when an intrinsic timescale is present. Handedness guided motor timing during externally-driven conditions that required cognitive intervention, which specifies the relevance of action expertise for the performance of timed-based motor activities. Overall, our results reveal that individual variation across domain-general and domain-specific levels of organization plays a steering role in how one predicts, perceives and experiences time, which accordingly impacts on cognition and behavior.

Manipulating Bodily Presence Affects Cross-Modal Spatial Attention: A Virtual-Reality-Based ERP Study.

Earlier studies have revealed cross-modal visuo-tactile interactions in endogenous spatial attention. The current research used event-related potentials (ERPs) and virtual reality (VR) to identify how the visual cues of the perceiver's body affect visuo-tactile interaction in endogenous spatial attention and at what point in time the effect takes place. A bimodal oddball task with lateralized tactile and visual stimuli was presented in two VR conditions, one with and one without visible hands, and one VR-free control with hands in view. Participants were required to silently count one type of stimulus and ignore all other stimuli presented in irrelevant modality or location. The presence of hands was found to modulate early and late components of somatosensory and visual evoked potentials. For sensory-perceptual stages, the presence of virtual or real hands was found to amplify attention-related negativity on the somatosensory N140 and cross-modal interaction in somatosensory and visual P200. For postperceptual stages, an amplified N200 component was obtained in somatosensory and visual evoked potentials, indicating increased response inhibition in response to non-target stimuli. The effect of somatosensory, but not visual, N200 enhanced when the virtual hands were present. The findings suggest that bodily presence affects sustained cross-modal spatial attention between vision and touch and that this effect is specifically present in ERPs related to early- and late-sensory processing, as well as response inhibition, but do not affect later attention and memory-related P3 activity. Finally, the experiments provide commeasurable scenarios for the estimation of the signal and noise ratio to quantify effects related to the use of a head mounted display (HMD). However, despite valid a-priori reasons for fearing signal interference due to a HMD, we observed no significant drop in the robustness of our ERP measurements.

Not My Problem: Vicarious Conflict Adaptation with Human and Virtual Co-actors.

The Simon effect refers to an incompatibility between stimulus and response locations resulting in a conflict situation and, consequently, slower responses. Like other conflict effects, it is commonly reduced after repetitions, suggesting an executive control ability, which flexibly rewires cognitive processing and adapts to conflict. Interestingly, conflict is not necessarily individually defined: the Social Simon effect refers to a scenario where two people who share a task show a conflict effect where a single person does not. Recent studies showed these observations might converge into what could be called vicarious conflict adaptation, with evidence indicating that observing someone else's conflict may subsequently reduce one's own. While plausible, there is reason for doubt: both the social aspect of the Simon Effect, and the degree to which executive control accounts for the conflict adaptation effect, have become foci of debate in recent studies. Here, we present two experiments that were designed to test the social dimension of the effect by varying the social relationship between the actor and the co-actor. In Experiment 1, participants performed a conflict task with a virtual co-actor, while the actor-observer relationship was manipulated as a function of the similarity between response modalities. In Experiment 2, the same task was performed both with a virtual and with a human co-actor, while heart-rate measurements were taken to measure the impact of observed conflict on autonomous activity. While both experiments replicated the interpersonal conflict adaptation effects, neither showed evidence of the critical social dimension. We consider the findings as demonstrating that vicarious conflict adaptation does not rely on the social relationship between the actor and co-actor.

Natural brain-information interfaces: Recommending information by relevance inferred from human brain signals.

Finding relevant information from large document collections such as the World Wide Web is a common task in our daily lives. Estimation of a user's interest or search intention is necessary to recommend and retrieve relevant information from these collections. We introduce a brain-information interface used for recommending information by relevance inferred directly from brain signals. In experiments, participants were asked to read Wikipedia documents about a selection of topics while their EEG was recorded. Based on the prediction of word relevance, the individual's search intent was modeled and successfully used for retrieving new relevant documents from the whole English Wikipedia corpus. The results show that the users' interests toward digital content can be modeled from the brain signals evoked by reading. The introduced brain-relevance paradigm enables the recommendation of information without any explicit user interaction and may be applied across diverse information-intensive applications.

The meaning of the virtual Midas touch: an ERP study in economic decision making.

The Midas touch refers to the altruistic effects of a brief touch. Though these effects have often been replicated, they remain poorly understood. We investigate the psychophysiology of the effect using remotely transmitted, precisely timed, tactile messages in an economic decision-making game called Ultimatum. Participants were more likely to accept offers after receiving a remotely transmitted touch. Furthermore, we found distinct effects of touch on event-related potentials evoked by (a) feedback regarding accepted and rejected offers, (b) decision cues related to proposals, and (c) the haptic and auditory cues themselves. In each case, a late positive effect of touch was observed and related to the P3. Given the role of the P3 in memory-related functions, the results indicate an indirect relationship between touch and generosity that relies on memory. This hypothesis was further tested and confirmed in the positive effects of touch on later proposals.

The self in conflict: actors and agency in the mediated sequential Simon task.

Executive control refers to the ability to withstand interference in order to achieve task goals. The effect of conflict adaptation describes that after experiencing interference, subsequent conflict effects are weaker. However, changes in the source of conflict have been found to disrupt conflict adaptation. Previous studies indicated that this specificity is determined by the degree to which one source causes episodic retrieval of a previous source. A virtual reality version of the Simon task was employed to investigate whether changes in a visual representation of the self would similarly affect conflict adaptation. Participants engaged in a mediated Simon task via 3D "avatar" models that either mirrored the participants' movements, or were presented statically. A retrieval cue was implemented as the identity of the avatar: switching it from a male to a female avatar was expected to disrupt the conflict adaptation effect (CAE). The results show that only in static conditions did the CAE depend on the avatar identity, while in dynamic conditions, changes did not cause disruption. We also explored the effect of conflict and adaptation on the degree of movement made with the task-irrelevant hand and replicated the reaction time pattern. The findings add to earlier studies of source-specific conflict adaptation by showing that a visual representation of the self in action can provide a cue that determines episodic retrieval. Furthermore, the novel paradigm is made openly available to the scientific community and is described in its significance for studies of social cognition, cognitive psychology, and human-computer interaction.

Hemispheric asymmetries and the control of motor sequences.

Sequencing of finger positions reflects a prototype of skilled behaviour. In order to perform sequencing, cognitive control supports the requirements and postural transitions. In this electroencephalography (EEG) study, we evaluate the effects of hand dominance and assess the neural correlates of unimanual and bimanual sequencing in left- and right-handers. The behavioural measurements provided an index of response planning (response time to first key press) and response execution (time between successive key presses, taps/s and percentage of correct responses), whereas the neural dynamics was determined by means of EEG coherence, expressing the functional connectivity between brain areas. Correlations between brain activity and behaviour were calculated for exploring the neural correlates that are functionally relevant for sequencing. Brain-behavioural correlations during response planning and execution revealed the significance of circuitry in the left hemisphere, underlining its significant role in the organisation of goal-directed behaviour. This lateralisation profile was independent of intrinsic constraints (hand dominance) and extrinsic demands (task requirements), suggesting essential higher-order computations in the left hemisphere. Overall, the observations highlight that the left hemisphere is specialised for sequential motor organisation in left- and right-handers, suggesting an endogenous hemispheric asymmetry for compound actions and the representation of skill; processes that can be separated from those that are involved in hand dominance.

Sequential modulations of the Simon effect depend on episodic retrieval.

Sequential modulations of conflict effects, like the reduction of the Simon effect after incompatible trials, have been taken to reflect the operation of a proactive control mechanism commonly called conflict monitoring. However, such modulations are often contaminated by episodic effects like priming and stimulus-response feature integration. It has previously been observed that if the episodic representation of a conflicting trial is altered by rotating the stimulus framing 180(∘) around its axis, the subsequent "conflict adaptation" pattern is eliminated. In Experiment 1, we replicate the findings and provide the basic episodic interpretation. In Experiment 2, we extend the framework to include rotations of 90(∘), and verify that the episodic effects generalize to scenarios of neutral compatibility. Finally, in Experiment 3, we add complete, 360(∘) rotations, and show that the episodic manipulation by itself does not eliminate the conflict adaptation patterns - as long as conditions favor episodic retrieval. The experiments are argued to demonstrate that an episodic account of the conflict adaptation effect can most parsimoniously account for the behavioral effects without relying on higher order cognition. Accordingly, we conclude that conflict adaptation can be understood either as critically depending on episodic retrieval, or alternatively reflecting only episodic retrieval itself.

Developmental changes in motor control: insights from bimanual coordination.

Manual dexterity is known to gradually progress with developmental age. In this study, we evaluate the performance of unimanual and bimanual actions under perturbed and unperturbed conditions in children between 4 and 10 years of age. Behavior was assessed by means of trajectory measurements and degree of bimanual coupling. The results showed that the younger children performed less successfully than the older children in the unimanual and bimanual tasks, with a plateau occurring around the age of 8 years. The SWAN rating scale of inattention and hyperactivity/impulsivity correlated with bimanual behavior at this particular age, suggesting that these traits are predictors of intricate skill performance during a critical developmental phase when significant refinement of control mechanisms occurs. The data furthermore revealed that a rebalancing of the between-hand performance asymmetries enabled superior bimanual coordination patterns in the older children. This suggests that progress in bimanual behavior relies on essential changes in unimanual processing and points to a dynamic interplay of circuitry. Overall, the data highlight a progressive change and integration of control systems due to developmental age with behavioral performance being guided by the existing constraints.