How Task Set and Task Switching Modulate Perceptual Processes: Is Recognition of Facial Emotion an Exception?

In Part 1 we review task-switching and other studies showing that, even with time for preparation, participants' ability to shift attention to a relevant attribute or object before the stimulus onset is limited: there is a 'residual cost'. In particular, several brain potential markers of perceptual encoding are delayed on task-switch trials, compared to task-repeat trials that require attention to the same attribute as before. Such effects have been documented even for a process often considered 'automatic' - visual word recognition: ERP markers of word frequency and word/nonword status are (1) delayed when the word recognition task follows a judgement of a perceptual property compared to repeating the lexical task, and (2) strongly attenuated during the perceptual judgements. Thus, even lexical access seems influenced by the task/attentional set. In Part 2, we report in detail a demonstration of what seems to be a special case, where task-set and a task switch have no such effect on perceptual encoding. Participants saw an outline letter superimposed on a face expressing neutral or negative emotion, and were auditorily cued to categorise the letter as vowel/consonant, or the face as emotional/neutral. ERPs exhibited a robust emotional-neutral difference (Emotional Expression Effect) no smaller or later when switching to the face task than when repeating it; in the first half of its time-course it did not vary with the task at all. The initial encoding of the valence of a fixated facial emotional expression appears to be involuntary and invariant, whatever the endogenous task/attentional set.

Serial or parallel proactive control of components of task-set? A task-switching investigation with concurrent EEG and eye-tracking.

Among the issues examined by studies of cognitive control in multitasking is whether processes underlying performance in the different tasks occur serially or in parallel. Here we ask a similar question about processes that pro-actively control task-set. In task-switching experiments, several indices of task-set preparation have been extensively documented, including anticipatory orientation of gaze to the task-relevant location (an unambiguous marker of reorientation of attention), and a positive polarity brain potential over the posterior cortex (whose functional significance is less well understood). We examine whether these markers of preparation occur in parallel or serially, and in what order. On each trial a cue required participants to make a semantic classification of one of three digits presented simultaneously, with the location of each digit consistently associated with one of three classification tasks (e.g., if the task was odd/even, the digit at the top of the display was relevant). The EEG positivity emerged following, and appeared time-locked to, the anticipatory fixation on the task-relevant location, which might suggest serial organisation. However, the fixation-locked positivity was not better defined than the cue-locked positivity; in fact, for the trials with the earliest fixations the positivity was better time-locked to the cue onset. This is more consistent with (re)orientation of spatial attention occurring in parallel with, but slightly before, the reconfiguration of other task-set components indexed by the EEG positivity.

Role of verbal working memory in rapid procedural acquisition of a choice response task.

How quickly are instructions for a task translated into an effective task-set? If declarative working memory (dWM) is used to maintain a task's S-R rules until practice compiles them adequately into procedural memory, variables that affect retrieval from dWM should influence task performance while it is still dependent on dWM. Participants were trained on a series of 6-choice RT tasks, with a 1:1 mapping from object pictures to keys. In Experiments 1 and 2, an instruction phase - presentation of the S-R rules one by one -was followed by test trials. The phonological similarity of the objects' names significantly affected performance only during the first few encounters with the stimuli. Serial position effects were also consistent with retrieval from verbal dWM during those early trials. In Experiment 3, instruction onon the S-R rules was omitted, so participants had to learn the S-R mappings by trial and error alone; the effect of phonological similarity lasted longer, but still disappeared after a dozen encounters with each stimulus. Experiment 4 showed that instructions and just four trials of practice per S-R rule were sufficient to form a persisting representation of the S-R rules robust enough to interfere with later acquisition of a competing S-R rule after several minutes spent acquiring other task-sets. An effective and lasting task-set is rapidly compiled into procedural memory through instruction and early feedback; verbal dWM plays little role thereafter.

PEP Does Not Dispense with but Implements Task-Set Reconfiguration. Can It Handle Phenomena More Diagnostic of Endogenous Control?

Invited commentary on Schmidt, Liefooghe, and De Houwer (2020) An episodic model of task switching effects: erasing the homunculus from memory. Journal of Cognition.

Can we prepare to attend to one of two simultaneous voices?

We can selectively attend to one of two simultaneous voices sharing a source location. Can we endogenously select the voice before speech is heard? Participants heard two digit names, spoken simultaneously by a male voice and a female voice, following a visual cue indicating which voice's digit to classify as odd or even. There was a substantial cost in reaction time and errors when the target voice switched from one trial to the next. In Experiment 1, with a highly familiar pair of voices, the switch cost reduced by nearly half as the cue-stimulus interval increased from 50 to 800 ms, indicating (contrary to previous reports) effective endogenous preparation for a change of voice. No further reduction in switch cost occurred with a longer preparation interval-this "residual" switch cost may be attributable to attentional "inertia." In Experiment 2, with previously unfamiliar voices, the pattern of switch costs was very similar, though repeated attention to the same target voice over a run of trials improved performance more. Delaying the onset of one voice by 366 ms improved performance, but the pattern of preparatory tuning effects was similar. Thus, endogenous preparation for a voice is possible, but it is limited in efficacy, as for some other attentional domains. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Mackintosh lecture-: Association and cognition: Two processes, one system.

This article argues that the dual-process position can be a useful first approximation when studying human mental life, but it cannot be the whole truth. Instead, we argue that cognition is built on association, in that associative processes provide the fundamental building blocks that enable propositional thought. One consequence of this position is to suggest that humans are able to learn associatively in a similar fashion to a rat or a pigeon, but another is that we must typically suppress the expression of basic associative learning in favour of rule-based computation. This stance conceptualises us as capable of symbolic computation but acknowledges that, given certain circumstances, we will learn associatively and, more importantly, be seen to do so. We present three types of evidence that support this position: The first is data on human Pavlovian conditioning that directly support this view. The second is data taken from task-switching experiments that provide convergent evidence for at least two modes of processing, one of which is automatic and carried out "in the background." And the last suggests that when the output of propositional processes is uncertain, the influence of associative processes on behaviour can manifest.

Is preparing for a language switch like preparing for a task switch?

A key index of top-down control in task switching-preparation for a switch-is underexplored in language switching. The well-documented EEG "signature" of preparation for a task switch-a protracted positive-polarity modulation over the posterior scalp-has thus far not been reported in language switching, and the interpretation of previously reported effects of preparation on language switching performance is complicated by confounding factors. In an experiment using event-related potentials (ERPs) and an optimized picture-naming paradigm that addressed these confounds the language was specified by an auditory cue on every trial and changed unpredictably. There were two key manipulations. First, the cue-stimulus interval allowed either generous (1,500 ms) or little (100 ms) opportunity for preparation. Second, to explore the interplay between bottom-up and top-down language selection, we compared a highly transparent and familiar "supercue"-the name of the language spoken in that language to a relatively opaque cue (short speeded-up fragment of national anthem). Preparation for a switch elicited a brain potential strongly reminiscent of the posterior switch positivity documented in task switching. As previously shown in task switching, its amplitude inversely predicted the performance "switch cost," demonstrated by our ERP analyses contingent on reaction time (RT). This overlap in the electrophysiological correlates of preparing to switch tasks and languages suggests domain-general processes for top-down selection of task-set and language for production. But, the surprisingly small language switch cost following the supercue in the short CSI suggests that rapid and (possibly automatic) bottom-up selection-not typically observed in task switching-may also occur. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Shifting Attention Between Visual Dimensions as a Source of Switch Costs.

Task-switching experiments have documented a puzzling phenomenon: Advance warning of the switch reduces but does not eliminate the switch cost. Theoretical accounts have posited that the residual switch cost arises when one selects the relevant stimulus-response mapping, leaving earlier perceptual processes unaffected. We put this assumption to the test by seeking electrophysiological markers of encoding a perceptual dimension. Participants categorized a colored letter as a vowel or consonant or its color as "warm" or "cold." Orthogonally to the color manipulation, some colors were eight times more frequent than others, and the letters were in upper- or lowercase. Color frequency modulated the electroencephalogram amplitude at around 150 ms when participants repeated the color-classification task. When participants switched from the letter task to the color task, this effect was significantly delayed. Thus, even when prepared for, a task switch delays or prolongs encoding of the relevant perceptual dimension.

Self-paced preparation for a task switch eliminates attentional inertia but not the performance switch cost.

The performance overhead associated with changing tasks (the "switch cost") usually diminishes when the task is specified in advance but is rarely eliminated by preparation. A popular account of the "residual" (asymptotic) switch cost is that it reflects "task-set inertia": carry-over of task-set parameters from the preceding trial(s). New evidence for a component of "task-set inertia" comes from eye-tracking, where the location associated with the previously (but no longer) relevant task is fixated preferentially over other irrelevant locations, even when preparation intervals are generous. Might such limits in overcoming task-set inertia in general, and "attentional inertia" in particular, result from suboptimal scheduling of preparation when the time available is outside one's control? In the present study, the stimulus comprised 3 digits located at the points of an invisible triangle, preceded by a central verbal cue specifying which of 3 classification tasks to perform, each consistently applied to just 1 digit location. The digits were presented only when fixation moved away from the cue, thus giving the participant control over preparation time. In contrast to our previous research with experimenter-determined preparation intervals, we found no sign of attentional inertia for the long preparation intervals. Self-paced preparation reduced but did not eliminate the performance switch cost-leaving a clear residual component in both reaction time and error rates. That the scheduling of preparation accounts for some, but not all, components of the residual switch cost, challenges existing accounts of the switch cost, even those which distinguish between preparatory and poststimulus reconfiguration processes. (PsycINFO Database Record

The coupling between spatial attention and other components of task-set: A task-switching investigation.

Is spatial attention reconfigured independently of, or in tandem with, other task-set components when the task changes? We tracked the eyes of participants cued to perform one of three digit-classification tasks, each consistently associated with a distinct location. Previously we observed, on task switch trials, a substantial delay in orientation to the task-relevant location and tendency to fixate the location of the previously relevant task-"attentional inertia". In the present experiments the cues specified (and instructions emphasized) the relevant location rather than the current task. In Experiment 1, with explicit spatial cues (arrows or spatial adverbs), the previously documented attentional handicaps all but disappeared, whilst the performance "switch cost" increased. Hence, attention can become decoupled from other aspects of task-set, but at a cost to the efficacy of task-set preparation. Experiment 2 used arbitrary single-letter cues with instructions and a training regime that encouraged participants to interpret the cue as indicating the relevant location rather than task. As in our previous experiments, and unlike in Experiment 1, we now observed clear switch-induced attentional delay and inertia, suggesting that the natural tendency is for spatial attention and task-set to be coupled and that only quasi-exogenous location cues decouple their reconfiguration.

A change of task prolongs early processes: evidence from ERPs in lexical tasks.

Switching tasks costs time. Allowing time to prepare reduces the cost, but usually leaves an irreducible "residual cost." Most accounts of this residual cost locate it within the response-selection stage of processing. To determine which processing stage is affected, we measured event-related potentials (ERPs) as participants performed a reading task or a perceptual judgment task, and examined the effect of a task switch on early markers of lexical processing. A task cue preceding a string of blue and red letters instructed the participant either to read the letter string (for a semantic classification in Experiment 1, and a lexical decision in Experiment 2) or to judge the symmetry of its color pattern. In Experiment 1, having to switch to the reading task delayed the evolution of the effect of word frequency on the reading task ERP by a substantial fraction of the effect on reaction time (RT). In Experiment 2, a task switch delayed the onset of the effect of lexical status on the ERP by about the same extent that it prolonged the RT. These effects indicate an early locus of (most of) the residual switch cost: We propose that this reflects a form of task-related attentional inertia. Other findings have implications for the automaticity of lexical access: Effects of frequency, lexicality, and orthographic familiarity on ERPs in the symmetry task indicated involuntary, but attenuated, orthographic and lexical processing even when attention was focused on a nonlexical property.

Is it harder to switch among a larger set of tasks?

When stimuli afford multiple tasks, switching among them involves promoting one of several task-sets in play into a most-active state. This process, often conceptualized as retrieving task parameters and stimulus-response (S-R) rules into procedural working memory, is a likely source of the reaction time (RT) cost of a task-switch, especially when no time is available for task preparation before the stimulus. We report 2 task-cuing experiments that asked whether the time consumed by task-set retrieval increases with the number of task-sets in play, while unconfounding the number of tasks with their frequency and recency of use. Participants were required to switch among 3 or 5 orthogonal classifications of perceptual attributes of an object (Experiment 1) or of phonological/semantic attributes of a word (Experiment 2), with a 100 or 1,300 ms cue-stimulus interval. For 2 tasks for which recency and frequency were matched in the 3- and 5-task conditions, there was no effect of number of tasks on the switch cost. For the other tasks, there was a greater switch cost in the 5-task condition with little time for preparation, attributable to effects of frequency/recency. Thus, retrieval time for active task-sets is not influenced by the number of alternatives per se (unlike several other kinds of memory retrieval) but is influenced by recency or frequency of use.

Attentional inertia and delayed orienting of spatial attention in task-switching.

Among the potential, but neglected, sources of task-switch costs is the need to reallocate attention to different attributes or objects. Even theorists who recognize the importance of attentional resetting in task-switching sometimes think it too efficient to result in significant behavioral costs. We examined the dynamics of spatial attention in a task-cuing paradigm using eye-tracking. Digits appeared simultaneously at 3 locations. A cue preceded this display by a variable interval, instructing the performance of 1 of 3 classification tasks (odd-even, low-high, inner-outer) each consistently associated with a location, so that task preparation could be tracked via fixation of the task-relevant location. Task-switching led to a delay in selecting the relevant location and a tendency to misallocate attention; the previously relevant location attracted attention much more than the other irrelevant location on switch trials, indicating "inertia" in attentional parameters rather than mere distractibility. These effects predicted reaction time switch costs within and over participants. The switch-induced delay was not confined to trials with slow/late orienting, but characteristic of most switch trials. The attentional pull of the previously relevant location was substantially reduced, but not eliminated, by extending the preparation interval to more than 1 sec, suggesting that attentional inertia contributes to the "residual" switch cost. A control condition, using identical displays but only 1 task, showed that these effects could not be attributed to the (small and transient) delays or inertia observed when the required orientation changed between trials in the absence of a task change.

Is performance in task-cuing experiments mediated by task set selection or associative compound retrieval?

Task-cuing experiments are usually intended to explore control of task set. But when small stimulus sets are used, they plausibly afford learning of the response associated with a combination of cue and stimulus, without reference to tasks. In 3 experiments we presented the typical trials of a task-cuing experiment: a cue (colored shape) followed, after a short or long interval, by a digit to which 1 of 2 responses was required. In a tasks condition, participants were (as usual) directed to interpret the cue as an instruction to perform either an odd/even or a high/low classification task. In a cue + stimulus → response (CSR) condition, to induce learning of mappings between cue-stimulus compound and response, participants were, in Experiment 1, given standard task instructions and additionally encouraged to learn the CSR mappings; in Experiment 2, informed of all the CSR mappings and asked to learn them, without standard task instructions; in Experiment 3, required to learn the mappings by trial and error. The effects of a task switch, response congruence, preparation, and transfer to a new set of stimuli differed substantially between the conditions in ways indicative of classification according to task rules in the tasks condition, and retrieval of responses specific to stimulus-cue combinations in the CSR conditions. Qualitative features of the latter could be captured by an associative learning network. Hence associatively based compound retrieval can serve as the basis for performance with a small stimulus set. But when organization by tasks is apparent, control via task set selection is the natural and efficient strategy.

TMS to V1 spares discrimination of emotive relative to neutral body postures.

This study used TMS to examine the role played by striate cortex (V1) in processing the emotional content of visual stimuli. Participants learned to discriminate two sets of body posture images. For half of each set, the posture's emotional significance (threat versus pleasant) provided a redundant cue for the discrimination; the other half were emotionally neutral. An image was briefly presented at a lateral location in the visual field where a TMS pulse produced a phosphene, or at a control location in the opposite hemifield. A TMS pulse 70-140 ms after stimulus presentation at the phosphene location impaired discrimination of neutral stimuli with little effect on discrimination of emotional stimuli; the two classes of stimuli were equally discriminable when presented at the control location. The results are consistent with the proposal that emotionally salient patterns, such as social threat, can be discriminated independently of the geniculo-striate pathway.

Are stimulus-response rules represented phonologically for task-set preparation and maintenance?

Accounts of task-set control generally assume that the current task's stimulus-response (S-R) rules must be elevated to a privileged state of activation. How are they represented in this state? In 3 task-cuing experiments, we tested the hypothesis that phonological working memory is used to represent S-R rules for task-set control by getting participants to switch between 2 sets of arbitrary S-R rules and manipulating the articulatory duration (Experiment 1) or phonological similarity (Experiments 2 and 3) of the names of the stimulus terms. The task cue specified which of 2 objects (Experiment 1) or consonants (Experiment 2) in a display to identify with a key press. In Experiment 3, participants switched between identifying an object/consonant and its color/visual texture. After practice, neither the duration nor the similarity of the stimulus terms had detectable effects on overall performance, task-switch cost, or its reduction with preparation. Only in the initial single-task training blocks was phonological similarity a significant handicap. Hence, beyond a very transient role, there is no evidence that (declarative) phonological working memory makes a functional contribution to representing S-R rules for task-set control, arguably because once learned, they are represented in nonlinguistic procedural working memory.

More attention to attention? An eye-tracking investigation of selection of perceptual attributes during a task switch.

Switching tasks prolongs response times, an effect reduced but not eliminated by active preparation. To explore the role of attentional selection of the relevant stimulus attribute in these task-switch costs, we measured eye fixations in participants cued to identify either a face or a letter displayed on its forehead. With only 200 ms between cue and stimulus onsets, the eyes fixated the currently relevant region of the stimulus less and the irrelevant region more on switch than on repeat trials, at stimulus onset and for 500 ms thereafter, in a pattern suggestive of delayed orientation of attention to the relevant region on switch trials. With 800 ms to prepare, both switch costs and inappropriate fixations were reduced, but on switch trials participants still tended (relative to repeat trials) to fixate the now-irrelevant region more at stimulus onset and to maintain fixation on, or refixate, the irrelevant region more during the next 500 ms. The size of this attentional persistence was associated with differences in performance costs between and within participants. We suggest that reorientation of attention is an important, albeit somewhat neglected and controversial, component of advance task-set reconfiguration and that the task-set inertia (or reactivation) to which many attribute the residual task-switch cost seen after preparation includes inertia in (or reactivation of) attentional parameters.

A brain-potential study of preparation for and execution of a task-switch with stimuli that afford only the relevant task.

Behavioural and neurophysiological studies of task-switching have tended to employ 'bivalent' stimuli (which afford responses in two tasks). Using brain potential recordings, we investigated task-switching with 'univalent' stimuli affording responses in only one of the tasks, and compared the outcomes to those recently obtained with bivalent stimuli (Lavric et al. [2008]: Eur J Neurosci 1-14), in order to examine two phenomena. First, when only univalent stimuli are presented, the processing of task cues becomes optional. Our results showed that in these circumstances linguistic (but not pictorial) cues were still effective in eliciting at least some degree of preparation for a task-switch, as evidenced by the reduction in the error cost of switching at the longer preparation interval and by a posterior switch-induced ERP positivity at about 450-800 ms in the cue-stimulus interval. Second, single affordance stimuli not only reduced behavioural switch costs relative to bivalent stimuli; they also produced a smaller post-stimulus switch-induced negativity, consistent with the latter being a marker of conflict between task-sets. However, using stimuli not associated with responses in the alternative task did not completely eliminate the negativity. We speculate that the residue reflects other sources of conflict: attention to the irrelevant perceptual dimension and/or persistence of task goals.

Neurophysiological signature of effective anticipatory task-set control: a task-switching investigation.

Changing between cognitive tasks requires a reorganization of cognitive processes. Behavioural evidence suggests this can occur in advance of the stimulus. However, the existence or detectability of an anticipatory task-set reconfiguration process remains controversial, in part because several neuroimaging studies have not detected extra brain activity during preparation for a task switch relative to a task repeat. In contrast, electrophysiological studies have identified potential correlates of preparation for a task switch, but their interpretation is hindered by the scarcity of evidence on their relationship to performance. We aimed to: (i) identify the brain potential(s) reflecting effective preparation for a task-switch in a task-cuing paradigm that shows clear behavioural evidence for advance preparation, and (ii) characterize this activity by means of temporal segmentation and source analysis. Our results show that when advance preparation was effective (as indicated by fast responses), a protracted switch-related component, manifesting itself as widespread posterior positivity and concurrent right anterior negativity, preceded stimulus onset for approximately 300 ms, with sources primarily in the left lateral frontal, right inferior frontal and temporal cortices. When advance preparation was ineffective (as implied by slow responses), or made impossible by a short cue-stimulus interval (CSI), a similar component, with lateral prefrontal generators, peaked approximately 300 ms poststimulus. The protracted prestimulus component (which we show to be distinct from P3 or contingent negative variation, CNV) also correlated over subjects with a behavioural measure of preparation. Furthermore, its differential lateralization for word and picture cues was consistent with a role for verbal self-instruction in preparatory task-set reconfiguration.

Can the task-cuing paradigm measure an endogenous task-set reconfiguration process?

In 6 task-cuing experiments, with 2 cues per task, the authors varied cue-stimulus interval to investigate G. D. Logan and C. Bundesen's (2003) claim that when cue repetition is controlled for, task-switch cost and its reduction with preparation are largely eliminated and hence cannot index an endogenous control process. Experiment 1 replicates their result, but Experiments 2 and 3, with similar designs, demonstrate a substantial task-switch cost, reducing with increasing cue-stimulus interval. Experiments 4 to 6 show that the critical difference is the probability of a task change: If it is kept low enough to discourage reconfiguration of task set unless and until the cue signals a task change, robust evidence for anticipatory task-set reconfiguration is obtained, even in Experiment 6, modeled closely on Logan and Bundesen's.