The Timing of Gaze Direction Perception: ERP Decoding and Task Modulation.

Distinguishing the direction of another person's eye gaze is extremely important in everyday social interaction, as it provides critical information about people's attention and, therefore, intentions. The temporal dynamics of gaze processing have been investigated using event-related potentials (ERPs) recorded with electroencephalography (EEG). However, the moment at which our brain distinguishes the gaze direction (GD), irrespectively of other facial cues, remains unclear. To solve this question, the present study aimed to investigate the time course of gaze direction processing, using an ERP decoding approach, based on the combination of a support vector machine and error-correcting output codes. We recorded EEG in young healthy subjects, 32 of them performing GD detection and 34 conducting face orientation tasks. Both tasks presented 3D realistic faces with five different head and gaze orientations each: 30°, 15° to the left or right, and 0°. While the classical ERP analyses did not show clear GD effects, ERP decoding analyses revealed that discrimination of GD, irrespective of head orientation, started at 140 ms in the GD task and at 120 ms in the face orientation task. GD decoding accuracy was higher in the GD task than in the face orientation task and was the highest for the direct gaze in both tasks. These findings suggest that the decoding of brain patterns is modified by task relevance, which changes the latency and the accuracy of GD decoding.

Effects of sensory modality and task relevance on omitted stimulus potentials.

Omitted stimulus potentials (OSPs) occur when a sensory stimulus is unexpectedly omitted. They are thought to reflect predictions about upcoming sensory events. The present study examined how OSPs differ across the sensory modalities of predicted stimuli. Twenty-nine university students were asked to press a mouse button at a regular interval of 1-2 s, which was immediately followed by either a visual or auditory stimulus in different blocks. The stimuli were sometimes omitted (p = 0.2), to which event-related potentials (ERPs) were recorded. The results showed that stimulus omissions in both modalities elicited ERP waveforms consisting of three components, oN1, oN2, and oP3. The peak latencies of these components were shorter in the auditory modality than in the visual modality. The amplitudes of OSPs were larger when participants were told that the omission indicated their poor performance (i.e., they pressed a button at an irregular interval) than when it was irrelevant to their performance. These findings suggest that OSPs occur from around 100 ms in a modality-specific manner and increase in amplitude depending on the task relevance of stimulus omissions.

Task relevance alters the effect of emotion on congruency judgments during action understanding.

The congruency judgments in action understanding helps individuals make timely adjustments to unexpected occurrence, and this process may be influenced by emotion. Previous research has showed contradictory effect of emotion on conflict processing, possibly due to the degree of relevance between emotion and task. However, to date, no study has systematically manipulated the relevance to explore how emotion affects congruency judgments in action understanding. We employed a cue-target paradigm and controlled the way emotional stimuli were presented on the target interface, setting up three experiments: emotion served as task-irrelevant distractor, task-irrelevant target and task-relevant target. The results showed that when emotion was irrelevant to the task, it impaired congruency judgements performance, regardless of a distractor or a target, while task-relevant emotion facilitated this process. These findings indicate that the impact of emotion on congruency judgements during action understanding depends on the degree of emotion-task relevance.

The modulation of task relevance on emotion-induced blindness depends on whether targets and distractors belong to the same category.

Previous research on emotion-induced blindness (EIB) argues emotional distractors capture attention in a bottom-up manner due to their physical and emotional salience. However, recent research has shown it is controversial whether EIB will be modulated by top-down factors. The present study further investigated whether the magnitude of EIB would be modulated by top-down factors, specifically the emotional relevance between tasks and distractors. Participants were divided into two groups having the same targets except for different task instructions. The orientation judgment group was asked to judge the orientation of the target (an emotionally irrelevant task), and the emotion judgment group was required to judge the emotional valence of the target (an emotionally relevant task). It was found the emotional relevance between tasks and distractors has no modulation on the magnitudes of EIB in two groups when targets and distractors are from different categories (Experiment 1), but a modulation when they are from the same category (Experiment 2). Consequently, we contend top-down task relevance modulates the EIB effect and distractors' priority is regulated by the emotional relevance between tasks and distractors. The current study holds attentional capture by stimulus-driven is unconditional in EIB, while attentional capture by goal-driven requires certain conditions.

Information processing in the dorsal pathway: from "where" to "what we do with it".

Visual processing in the brain has been understood as the ventral and dorsal pathways processing "what" and "where" information, respectively. Mocz et al. (Mocz V, Vaziri-Pashkam M, Chun M, Xu Y. J Cogn Neurosci 34: 2406-2435, 2022), however, report that the two pathways code object features in a parallel manner. These results support that information processing in the dorsal pathway is not strictly limited to "where" and that the two pathways work in parallel to process task-relevant information ("what we do with it").

Top-Down Modulation of Early Visual Processing in V1: Dissociable Neurophysiological Effects of Spatial Attention, Attentional Load and Task-Relevance.

Until today, there is an ongoing discussion if attention processes interact with the information processing stream already at the level of the C1, the earliest visual electrophysiological response of the cortex. We used two highly powered experiments (each N = 52) and examined the effects of task relevance, spatial attention, and attentional load on individual C1 amplitudes for the upper or lower visual hemifield. Bayesian models revealed evidence for the absence of load effects but substantial modulations by task-relevance and spatial attention. When the C1-eliciting stimulus was a task-irrelevant, interfering distracter, we observed increased C1 amplitudes for spatially unattended stimuli. For spatially attended stimuli, different effects of task-relevance for the two experiments were found. Follow-up exploratory single-trial analyses revealed that subtle but systematic deviations from the eye-gaze position at stimulus onset between conditions substantially influenced the effects of attention and task relevance on C1 amplitudes, especially for the upper visual field. For the subsequent P1 component, attentional modulations were clearly expressed and remained unaffected by these deviations. Collectively, these results suggest that spatial attention, unlike load or task relevance, can exert dissociable top-down modulatory effects at the C1 and P1 levels.

Facilitation of tactile processing during action observation of goal-directed reach and grasp movements.

Our perception of sensory events can be altered by action, but less is known about how our perception can be altered by action observation. For example, our ability to detect tactile stimuli is reduced when our limb is moving, and task-relevance and movement speed can alter such tactile detectability. During action observation, however, the relationship between tactile processing and such modulating factors is not known. Thus, the current study sought to explore tactile processing at a task-relevant location during the observation of reaching and grasping movements performed at different speeds. Specifically, participants observed videos of an anonymous model performing movements at a slow [i.e., peak velocity (PV): 155 mm/s], medium (i.e., PV: 547 mm/s), or fast speed (i.e., PV: 955 mm/s). To assess tactile processing, weak electrical stimuli of different amplitudes were presented to participants' right thumbs when the observed model was at their starting position and prior to any movement, or when the observed model's limb reached its PV. When observing slow movements, normalized perceptual thresholds were significantly lower/better than for the premovement stimulation time. These data suggest that the movement speed can modulate tactile processing, even when observing a movement. Furthermore, these findings provide seminal evidence for tactile facilitation at a task-relevant location during the observation of slow reaching and grasping movements (i.e., speeds associated with tactile exploration).NEW & NOTEWORTHY Previous work has highlighted the relationship between touch processing and movement speed during action, but the current study sought to understand this relationship during action observation of reaching and grasping movements. Here, we provide seminal evidence that tactile perceptual thresholds at the thumb are reduced compared with rest when observing the slowest movement speeds. Thus, tactile processing was facilitated at a task-relevant location during the observation of movements with speeds associated with tactile exploration.

Comparing the temporal dynamics and efficacy of task-relevant and task-irrelevant memory-driven attention.

People's attention is well attracted to a stimulus matching their memory. For example, when people are required to remember the color of a visual object, stimuli matching the memory color powerfully capture attention. Remarkably, stimuli with the shape of the memory object, that is, irrelevant-matching stimuli were also found to capture attention. Here, we examined how task relevance affects the temporal dynamics and the strength of memory-driven attention. In the experiment, participants performed a visual search task while maintaining the color or shape of a colored shape. When participants were required to memorize the color of the memory sample, the shape of the sample stimulus is task-irrelevant feature and vice versa. Importantly, while a search item matching working memory in the task-relevant dimension was presented for one group of participants, an irrelevant-matching search item appeared for the other group of participants. Further, we varied stimulus onset asynchrony (SOA) between the memory sample and search items. We found that relevant-matching stimuli captured attention regardless of whether the SOA was short or long. However, attentional capture by irrelevant-matching stimuli depended on the SOA; no memory-driven capture was observed at the shortest SOA, but significant capture was found at longer SOAs. Further, the capture effects by relevant-matching stimuli were greater than that of irrelevant-matching stimuli. These findings suggest both task-relevant and -irrelevant features in working memory affect the attentional selection in visual search task, but its temporal dynamics and strength are modulated by the task-relevance.

Dissociating neural correlates of consciousness and task relevance during auditory processing.

In recent years, several ERP components have been identified as potential neural correlates of consciousness (NCC), including early negativities and late positivities. Based on experiments in the visual modality, it has recently been shown that awareness is often confounded with reporting it, possibly overestimating the NCC. It is unknown whether similar constraints also exist in the auditory modality. In order to address this gap, we presented spoken words in a sustained inattentional deafness paradigm. Electrophysiological responses were obtained in three physically identical experimental conditions that differed only with respect to the participants' instructions. Participants were either left uninformed or informed about the presence of spoken words while confronted with an auditory distractor task (U/I condition), informed about the words while exposed to the same task as before (I condition), or requested to respond to the now task-relevant speech stimuli (TR condition). After completion of the U/I condition, only informed participants reported awareness of the words. In ERPs, awareness of words in the U/I and I condition was accompanied by an anterior auditory awareness negativity (AAN). Only when stimuli were task-relevant, i.e., during the TR condition, late positivities emerged. Taken together, these results indicate that early negativities but not late positivities index awareness across sensory modalities. Thus, they provide evidence for a recurrent processing framework, which highlights the importance of early sensory processing in conscious perception.

The effect of threatening facial expressions on inhibition-induced forgetting depends on their task-relevance.

Inhibition-induced forgetting refers to impaired memory for the stimuli to which responses were inhibited. The present study aimed to examine if it would be modulated by the processing of threatening facial expressions. Angry and neutral faces were presented in a go/no-go task and subsequent memory for faces was measured in a surprise recognition task. In Experiment 1, task-irrelevant angry and neutral faces appeared randomly, and participants responded to the gender of the faces during the go/no-go task. Results showed that the perception of neutral faces was possibly biased by angry faces. So, in Experiment 2, angry and neutral faces were given in separate blocks while participants still responded to the gender. Inhibition-induced forgetting was not modulated by facial expressions, as it was observed for both angry and neutral faces. Finally, in Experiment 3, where participants were assigned to respond to either angry or neutral faces, so that facial expressions were relevant, inhibition-induced forgetting was negated only in the group in whom responses to angry faces were inhibited. The findings suggest that task-relevance plays a key role in the way the processing of emotional information influences the interaction between cognitive control and memory encoding.

The contribution of the prefrontal cortex to relevancy-based gating of visual and tactile stimuli.

Patients with lesions of the prefrontal cortex (PFC) show increased distractibility and impairments in inhibiting cortical responses to irrelevant stimuli. This study was designed to test the role of the PFC in the early modality-specific modulation of event-related potentials (ERPs) generated during a sensory selection task. The task required participants to make a scaled motor response to the amplitudes of visual and tactile stimuli presented individually or concurrently. Task relevance was manipulated and continuous theta burst stimulation (cTBS) was used to transiently inhibit PFC activity to test the contribution of the PFC to modulation of sensory gating. Electroencephalography (EEG) was collected from participants both before and after cTBS was applied. The somatosensory-evoked N70 ERP was shown to be modulated by task relevance before but not after cTBS was applied to the PFC, and downregulating PFC activity through the use of cTBS abolished any relevancy differences in N70 amplitude. In conclusion, this study demonstrated that early modality-specific changes in cortical somatosensory processing are modulated by attention, and that this effect is subserved by prefrontal cortical activity.

Threat-induced impulsivity in Go/Nogo tasks: Relationships to task-relevance of emotional stimuli and virtual proximity.

Threatening stimuli are thought to induce impulsive responses, but Emotional Go/Nogo task results are not in line with this. We extend previous research by testing effects of task-relevance of emotional stimuli and virtual proximity. Four studies were performed to test this in healthy college students. When emotional stimuli were task-relevant, threat both increased commission errors and decreased RT, but this was not found when emotional stimuli were task-irrelevant. This was found in both between-subject and within-subject designs. These effects were found using a task version with equal go and nogo rates, but not with 90-10% go-nogo rates. Proximity was found to increase threat-induced speeding, with task-relevant stimuli only, although effects on accuracy were less clear. Threat stimuli can thus induce impulsive responding, but effects depend on features of the task design. The results may be of use in understanding theoretically unexpected results involving threat and impulsivity and designing future studies.

The processing of task-irrelevant emotion and colour in the Approach-Avoidance Task.

When processing information about human faces, we have to integrate different sources of information like skin colour and emotional expression. In 3 experiments, we investigated how these features are processed in a top-down manner when task instructions determine the relevance of features, and in a bottom-up manner when the stimulus features themselves determine process priority. In Experiment 1, participants learned to respond with approach-avoidance movements to faces that presented both emotion and colour features (e.g. happy faces printed in greyscale). For each participant, only one of these two features was task-relevant while the other one could be ignored. In contrast to our predictions, we found better learning of task-irrelevant colour when emotion was task-relevant than vice versa. Experiment 2 showed that the learning of task-irrelevant emotional information was improved in general when participants' awareness was increased by adding NoGo-trials. Experiment 3 replicated these results for faces and emotional words. We conclude that during the processing of faces, both bottom-up and top-down processes are involved, such that task instructions and feature characteristics play a role. Ecologically significant features like emotions are not necessarily processed with high priority. The findings are discussed in the light of theories of attention and cognitive biases.

Delayed reconfiguration of a non-emotional task set through reactivation of an emotional task set in task switching: an ageing study.

In our everyday life, we frequently switch between different tasks, a faculty that changes with age. However, it is still not understood how emotion impacts on age-related changes in task switching. Using faces with emotional and neutral expressions, Experiment 1 investigated younger (n = 29; 18-38 years old) and older adults' (n = 32; 61-80 years old) ability to switch between an emotional and a non-emotional task (i.e. responding to the face's expression vs. age). In Experiment 2, younger and older adults also viewed emotional and neutral faces, but switched between two non-emotional tasks (i.e. responding to the face's age vs. gender). Data from Experiment 1 demonstrated that switching from an emotional to a non-emotional task was slower when the expression of the new face was emotional rather than neutral. This impairment was observed in both age groups. In contrast, Experiment 2 revealed that neither younger nor older adults were affected by block-wise irrelevant emotion when switching between two non-emotional tasks. Overall, the findings suggest that task-irrelevant emotion can impair task switching through reactivation of the competing emotional task set. They also suggest that this effect and the ability to shield task-switching performance from block-wise irrelevant emotion are preserved in ageing.

Reexamining unconscious response priming: A liminal-prime paradigm.

Research on the limits of unconscious processing typically relies on the subliminal-prime paradigm. However, this paradigm is limited in the issues it can address. Here, we examined the implications of using the liminal-prime paradigm, which allows comparing unconscious and conscious priming with constant stimulation. We adapted an iconic demonstration of unconscious response priming to the liminal-prime paradigm. On the one hand, temporal attention allocated to the prime and its relevance to the task increased the magnitude of response priming. On the other hand, the longer RTs associated with the dual task inherent to the paradigm resulted in response priming being underestimated, because unconscious priming effects were shorter-lived than conscious-priming effects. Nevertheless, when the impact of long RTs was alleviated by considering the fastest trials or by imposing a response deadline, conscious response priming remained considerably larger than unconscious response priming. These findings suggest that conscious perception strongly modulates response priming.

Occipital, parietal, and frontal cortices selectively maintain task-relevant features of multi-feature objects in visual working memory.

Previous studies have shown that information held in visual working memory is represented in the occipital, parietal, and frontal cortices. However, less is known about whether the mnemonic information of multi-feature objects is modulated by task demand in the parietal and frontal regions. To address this question, we asked participants to remember either color or orientation of one of the two colored gratings for a delay. Using fMRI and an inverted encoding model, we reconstructed population-level, feature-selective responses in the occipital, parietal and frontal cortices during memory maintenance. We found that not only orientation but also color information can be maintained in higher-order parietal and frontal cortices as well as the early visual cortex when it was cued to be remembered. Conversely, neither the task-irrelevant feature of the cued object, nor any feature of the uncued object was maintained in the occipital, parietal, or frontal cortices. These results suggest a highly selective mechanism of visual working memory that maintains task-relevant features only.

Reach-relevant somatosensory signals modulate tactile suppression.

Tactile stimuli on moving limbs are typically attenuated during reach planning and execution. This phenomenon has been related to internal forward models that predict the sensory consequences of a movement. Tactile suppression is considered to occur due to a match between the actual and predicted sensory consequences of a movement, which might free capacities to process novel or task-relevant sensory signals. Here, we examined whether and how tactile suppression depends on the relevance of somatosensory information for reaching. Participants reached with their left or right index finger to the unseen index finger of their other hand (body target) or an unseen pad on a screen (external target). In the body target condition, somatosensory signals from the static hand were available for localizing the reach target. Vibrotactile stimuli were presented on the moving index finger before or during reaching or in a separate no-movement baseline block, and participants indicated whether they detected a stimulus. As expected, detection thresholds before or during reaching were higher compared with baseline. Tactile suppression was also stronger for reaches to body targets than external targets, as reflected by higher detection thresholds and lower precision of detectability. Moreover, detection thresholds were higher when reaching with the left than with the right hand. Our results suggest that tactile suppression is modulated by position signals from the target limb that are required to reach successfully to the own body. Moreover, limb dominance seems to affect tactile suppression, presumably due to disparate uncertainty of feedback signals from the moving limb.NEW & NOTEWORTHY Tactile suppression on a moving limb has been suggested to release computational resources for processing other relevant sensory events. In the current study, we show that tactile sensitivity on the moving limb decreases more when reaching to body targets than external targets. This indicates that tactile perception can be modulated by allocating processing capacities to movement-relevant somatosensory information at the target location. Our results contribute to understanding tactile processing and predictive mechanisms in the brain.

From reaching to reach-to-grasp: the arm posture difference and its implications on human motion control strategy.

Reach-to-grasp arm postures differ from those in pure reaching because they are affected by grasp position/orientation, rather than simple transport to a position during a reaching motion. This paper investigates this difference via an analysis of experimental  data collected on reaching and reach-to-grasp motions. A seven-degree-of-freedom (DOFs) kinematic arm model with the swivel angle is used for the motion analysis. Compared to a widely used anatomical arm model, this model distinguishes clearly the four grasping-relevant DOFs (GR-DOFs) that are affected by positions and orientations of the objects to be grasped. These four GR-DOFs include the swivel angle that measures the elbow rotation about the shoulder-wrist axis, and three wrist joint angles. For each GR-DOF, we quantify position vs orientation task-relevance bias that measures how much the DOF is affected by the grasping position vs orientation. The swivel angle and forearm supination have similar bias, and the analysis of their motion suggests two hypotheses regarding the synergistic coordination of the macro- and micro-structures of the human arm (1) DOFs with similar task-relevance are synergistically coordinated; and (2) such synergy breaks when a task-relevant DOF is close to its joint limit without necessarily reaching the limit. This study provides a motion analysis method to reduce the control complexity for reach-to-grasp tasks, and suggests using dynamic coupling to coordinate the hand and arm of upper-limb exoskeletons.

ERP signatures of conscious and unconscious word and letter perception in an inattentional blindness paradigm.

A three-phase inattentional blindness paradigm was combined with ERPs. While participants performed a distracter task, line segments in the background formed words or consonant-strings. Nearly half of the participants failed to notice these word-forms and were deemed inattentionally blind. All participants noticed the word-forms in phase 2 of the experiment while they performed the same distracter task. In the final phase, participants performed a task on the word-forms. In all phases, including during inattentional blindness, word-forms elicited distinct ERPs during early latencies (∼200-280ms) suggesting unconscious orthographic processing. A subsequent ERP (∼320-380ms) similar to the visual awareness negativity appeared only when subjects were aware of the word-forms, regardless of the task. Finally, word-forms elicited a P3b (∼400-550ms) only when these stimuli were task-relevant. These results are consistent with previous inattentional blindness studies and help distinguish brain activity associated with pre- and post-perceptual processing from correlates of conscious perception.

Decoding of task-relevant and task-irrelevant intracranial EEG representations.

Natural stimuli consist of multiple properties. However, not all of these properties are equally relevant in a given situation. In this study, we applied multivariate classification algorithms to intracranial electroencephalography data of human epilepsy patients performing an auditory Stroop task. This allowed us to identify neuronal representations of task-relevant and irrelevant pitch and semantic information of spoken words in a subset of patients. When properties were relevant, representations could be detected after about 350ms after stimulus onset. When irrelevant, the association with gamma power differed for these properties. Patients with more reliable representations of irrelevant pitch showed increased gamma band activity (35-64Hz), suggesting that attentional resources allow an increase in gamma power in some but not all patients. This effect was not observed for irrelevant semantics, possibly because the more automatic processing of this property allowed for less variation in free resources. Processing of different properties of the same stimulus seems therefore to be dependent on the characteristics of the property.