Moving visual world experiments online? A web-based replication of Dijkgraaf, Hartsuiker, and Duyck (2017) using PCIbex and WebGazer.js.

The visual world paradigm is one of the most influential paradigms to study real-time language processing. The present study tested whether visual world studies can be moved online, using PCIbex software (Zehr & Schwarz, 2018) and the WebGazer.js algorithm (Papoutsaki et al., 2016) to collect eye-movement data. Experiment 1 was a fixation task in which the participants looked at a fixation cross in multiple positions on the computer screen. Experiment 2 was a web-based replication of a visual world experiment by Dijkgraaf et al. (2017). Firstly, both experiments revealed that the spatial accuracy of the data allowed us to distinguish looks across the four quadrants of the computer screen. This suggest that the spatial resolution of WebGazer.js is fine-grained enough for most visual world experiments (which typically involve a two-by-two quadrant-based set-up of the visual display). Secondly, both experiments revealed a delay of roughly 300 ms in the time course of the eye movements, possibly caused by the internal processing speed of the browser or WebGazer.js. This delay can be problematic in studying questions that require a fine-grained temporal resolution and requires further investigation.

Does passive sound attenuation affect responses to pitch-shifted auditory feedback?

The role of auditory feedback in vocal production has mainly been investigated by altered auditory feedback (AAF) in real time. In response, speakers compensate by shifting their speech output in the opposite direction. Current theory suggests this is caused by a mismatch between expected and observed feedback. A methodological issue is the difficulty to fully isolate the speaker's hearing so that only AAF is presented to their ears. As a result, participants may be presented with two simultaneous signals. If this is true, an alternative explanation is that responses to AAF depend on the contrast between the manipulated and the non-manipulated feedback. This hypothesis was tested by varying the passive sound attenuation (PSA). Participants vocalized while auditory feedback was unexpectedly pitch shifted. The feedback was played through three pairs of headphones with varying amounts of PSA. The participants' responses were not affected by the different levels of PSA. This suggests that across all three headphones, PSA is either good enough to make the manipulated feedback dominant, or differences in PSA are too small to affect the contribution of non-manipulated feedback. Overall, the results suggest that it is important to realize that non-manipulated auditory feedback could affect responses to AAF.

Syntactic levels, lexicalism, and ellipsis: The jury is still out.

Structural priming data are sometimes compatible with several theoretical views, as shown here for three key theoretical claims. One reason is that prime sentences affect multiple representational levels driving syntactic choice. Additionally, priming is affected by further cognitive functions (e.g., memory). We therefore see priming as a useful tool for the investigation of linguistic representation but not the only tool.

Conflict monitoring in speech processing: An fMRI study of error detection in speech production and perception.

To minimize the number of errors in speech, and thereby facilitate communication, speech is monitored before articulation. It is, however, unclear at which level during speech production monitoring takes place, and what mechanisms are used to detect and correct errors. The present study investigated whether internal verbal monitoring takes place through the speech perception system, as proposed by perception-based theories of speech monitoring, or whether mechanisms independent of perception are applied, as proposed by production-based theories of speech monitoring. With the use of fMRI during a tongue twister task we observed that error detection in internal speech during noise-masked overt speech production and error detection in speech perception both recruit the same neural network, which includes pre-supplementary motor area (pre-SMA), dorsal anterior cingulate cortex (dACC), anterior insula (AI), and inferior frontal gyrus (IFG). Although production and perception recruit similar areas, as proposed by perception-based accounts, we did not find activation in superior temporal areas (which are typically associated with speech perception) during internal speech monitoring in speech production as hypothesized by these accounts. On the contrary, results are highly compatible with a domain general approach to speech monitoring, by which internal speech monitoring takes place through detection of conflict between response options, which is subsequently resolved by a domain general executive center (e.g., the ACC).

Can Faces Prime a Language?

Bilinguals have two languages that are activated in parallel. During speech production, one of these languages must be selected on the basis of some cue. The present study investigated whether the face of an interlocutor can serve as such a cue. Spanish-Catalan and Dutch-French bilinguals were first familiarized with certain faces, each of which was associated with only one language, during simulated Skype conversations. Afterward, these participants performed a language production task in which they generated words associated with the words produced by familiar and unfamiliar faces displayed on-screen. When responding to familiar faces, participants produced words faster if the faces were speaking the same language as in the previous Skype simulation than if the same faces were speaking a different language. Furthermore, this language priming effect disappeared when it became clear that the interlocutors were actually bilingual. These findings suggest that faces can prime a language, but their cuing effect disappears when it turns out that they are unreliable as language cues.

Sex Differences in Neurophysiological Activation Patterns During Phonological Input Processing: An Influencing Factor for Normative Data.

In the context of neurophysiological normative data, it has been established that aging has a significant impact on neurophysiological correlates of auditory phonological input processes, such as phoneme discrimination (PD) and word recognition (WR). Besides age, sex is another demographic factor that influences several language processes. We aimed to disentangle whether sex has a similar effect on PD and WR. Event-related potentials (ERPs) were recorded in 20 men and 24 women. During PD, three phonemic contrasts (place and manner of articulation and voicing) were compared using the attentive P300 and pre-attentive Mismatch Negativity. To investigate WR, real words were contrasted with pseudowords in a pre-attentive oddball task. Women demonstrated a larger sensitivity to spectrotemporal differences, as evidenced by larger P300 responses to the place of articulation (PoA) contrast and larger P300 and MMN responses than men in PoA-based PD. Men did not display such sensitivity. Attention played an important role, considering that women needed more attentional resources to differentiate between PoA and the other phonemic contrasts. During WR, pseudowords evoked larger amplitudes already 100 ms post-stimulus independent of sex. However, women had decreased P200 latencies, but longer N400 latencies in response to pseudowords, whereas men showed increased N400 latencies compared to women in response to real words. The current results demonstrate significant sex-related influences on phonological input processes. Therefore, existing neurophysiological normative data for age should be complemented for the factor sex.

The effects of discourse coherence on the persistence of sentence structures.

We studied the role of discourse coherence relations on structure formulation in sentence production by examining whether a connective, an essential signal of coherence relations, modulates the tendency for speakers to reuse sentence structures (i.e., structural priming). We further examined three possible modulating factors: the type of connectives (additive vs. adversative connective), event similarity (similar event vs. different event), and topic cohesion (with or without available anaphoric antecedent). In four structural priming experiments, native Dutch participants were asked to read either a Dutch double object sentence or a prepositional object sentence and describe pictures that depicted ditransitive events. Critically, the prime and the target either were linked by a connective (en "and" or maar "but") or were not linked. The verb overlap between the prime and the target was also manipulated. In Experiment 1, the presence of en facilitated structural priming, but only when the verbs were different. In Experiment 2, maar reduced structural priming when the verbs were repeated. Experiment 3 replicated the findings of Experiments 1 and 2 in a within-subjects design. In Experiment 4, there was no referential link between the sentences. Now there was no connective effect on structural priming. Taken together, we demonstrated that the insertion of a connective influences syntactic persistence. The connective effects vary across semantic properties of the connectives, event similarity, and referential continuity, suggesting that the production of sentence structure is modulated by speakers' prediction about listeners' inference of coherence relations between consecutive utterances. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Are forward models enough to explain self-monitoring? Insights from patients and eye movements.

At the core of Pickering & Garrod's (P&G's) theory is a monitor that uses forward models. I argue that this account is challenged by neuropsychological findings and visual world eye-tracking data and that it has two conceptual problems. I propose that conflict monitoring avoids these issues and should be considered a promising alternative to perceptual loop and forward modeling theories.

Structure prediction occurs when it is needed: Evidence from visual-world structural priming in Dutch comprehension.

Arai et al. (2007) showed that structural priming in the comprehension of English dative sentences only occurred when the verb was repeated between prime and target, suggesting a lexically-dependent mechanism of structure prediction. However, a recent study in Mandarin comprehension found abstract (verb-independent) structural priming and such priming was stronger when the structure was unexpected (e.g., when a verb biased toward the double object [DO] structure is followed by an unexpected prepositional object [PO] structure; Chen et al., 2022). The latter finding of inverse preference priming is consistent with error-based implicit learning accounts, which suggest structural priming is based on learning from prediction errors (Chang et al., 2006). Here we tested the mechanism of structure prediction (lexically-dependent vs. abstract) in four visual-world comprehension experiments in Dutch. Dutch is a Germanic language like English; it is biased toward the PO structure like Mandarin. We not only found structural priming when the verb was repeated, but also when the verb was different: During target sentence processing, comprehenders looked more often at the recipient (predicting a DO structure) than at the theme (predicting a PO structure) after a DO prime and vice versa after a PO prime. Importantly, abstract structural priming only occurred when the target structure was relatively unpredictable. We interpret the inconsistent findings across languages in terms of an effortful process of structure prediction in comprehension (Pickering & Gambi, 2018): it occurs when it is needed to disambiguate the postverbal arguments, but not when it is optional. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Can object identification difficulty be predicted based on disfluencies and eye-movements in connected speech?

In the current study, we asked whether delays in the earliest stages of picture naming elicit disfluency. To address this question, we used a network task, where participants describe the route taken by a marker through visually presented networks of objects. Additionally, given that disfluencies are arguably multifactorial, we combined this task with eye tracking, to be able to disentangle disfluency related to word preparation from other factors (e.g., stalling strategy). We used visual blurring, which hinders visual identification of the items and thereby slows down selection of a lexical concept. We tested the effect of this manipulation on disfluency production and visual attention. Blurriness did not lead to more disfluency on average and viewing times decreased with blurred pictures. However, multivariate pattern analyses revealed that a classifier could predict above chance, from the pattern of disfluency, whether each participant was about to name blurred or control pictures. Impeding the conceptual generation of a message therefore affected the pattern of disfluencies of each participant individually, but this pattern was not consistent from one participant to another. Additionally, some of the disfluency and eye-movement variables correlated with individual cognitive differences, in particular with inhibition.

Semantic interference affects speech production by increasing disfluencies, not errors.

Several studies have shown that different types of disfluency occur depending on the language production stage at which people experience difficulties. The current study combined a network task and a picture-word interference task to analyse whether lexical-semantic difficulty triggers errors and disfluencies in connected-speech production. The participants produced more disfluencies in the presence of a semantically related distractor word than an unrelated distractor word, while few semantic errors were made. These results support the hypothesis that difficulties at distinct stages of language production lead to distinct patterns of disfluency, with lexical-semantic difficulties leading to self-corrections and silent pauses. The results also have implications for the role of the monitoring system in connected-speech production.

Purposeful listening in challenging conditions: A study of prediction during consecutive interpreting in noise.

Prediction is often used during language comprehension. However, studies of prediction have tended to focus on L1 listeners in quiet conditions. Thus, it is unclear how listeners predict outside the laboratory and in specific communicative settings. Here, we report two eye-tracking studies which used a visual-world paradigm to investigate whether prediction during a consecutive interpreting task differs from prediction during a listening task in L2 listeners, and whether L2 listeners are able to predict in the noisy conditions that might be associated with this communicative setting. In a first study, thirty-six Dutch-English bilinguals either just listened to, or else listened to and then consecutively interpreted, predictable sentences presented on speech-shaped sound. In a second study, another thirty-six Dutch-English bilinguals carried out the same tasks in clear speech. Our results suggest that L2 listeners predict the meaning of upcoming words in noisy conditions. However, we did not find that predictive eye movements depended on task, nor that L2 listeners predicted upcoming word form. We also did not find a difference in predictive patterns when we compared our two studies. Thus, L2 listeners predict in noisy circumstances, supporting theories which posit that prediction regularly takes place in comprehension, but we did not find evidence that a subsequent production task or noise affects semantic prediction.

Don't blame yourself: Conscious source monitoring modulates feedback control during speech production.

Sensory feedback plays an important role in speech motor control. One of the main sources of evidence for this is studies in which online auditory feedback is perturbed during ongoing speech. In motor control, it is therefore crucial to distinguish between sensory feedback and externally generated sensory events. This is called source monitoring. Previous altered feedback studies have taken non-conscious source monitoring for granted, as automatic responses to altered sensory feedback imply that the feedback changes are processed as self-caused. However, the role of conscious source monitoring is unclear. The current study investigated whether conscious source monitoring modulates responses to unexpected pitch changes in auditory feedback. During the first block, some participants spontaneously attributed the pitch shifts to themselves (self-blamers) while others attributed them to an external source (other-blamers). Before Block 2, all participants were informed that the pitch shifts were experimentally induced. The self-blamers then showed a reduction in response magnitude in Block 2 compared with Block 1, while the other-blamers did not. This suggests that conscious source monitoring modulates responses to altered auditory feedback, such that consciously ascribing feedback to oneself leads to larger compensation responses. These results can be accounted for within the dominant comparator framework, where conscious source monitoring could modulate the gain on sensory feedback. Alternatively, the results can be naturally explained from an inferential framework, where conscious knowledge may bias the priors in a Bayesian process to determine the most likely source of a sensory event.

Error-based structure prediction in language comprehension: Evidence from verb bias effects in a visual-world structural priming paradigm for Mandarin Chinese.

Structural priming studies in production have demonstrated stronger priming effects for unexpected sentence structures (inverse preference effect). This is consistent with error-based implicit learning accounts that assume learning depends on prediction error. Such prediction error can be verb-specific, leading to strong priming when a verb that is for instance biased toward the prepositional object (PO) structure occurs with an unexpected double object (DO) structure. However, it is unclear whether this mechanism also holds for language comprehension, especially for languages like Mandarin Chinese, which arguably depends strongly on semantics to predict syntax in comprehension. Experiment 1 was a norming study (N = 367) that measured the biases (DO vs. PO) of 48 Mandarin Chinese dative verbs. Experiment 2 (N = 72) crossed verb bias (DO-bias or PO-bias) and structure (DO or PO) of prime sentences in a visual-world paradigm, to examine whether Mandarin comprehenders show an inverse preference effect. The priming effect is expressed as the proportion of looks to the predicted referent (i.e., the recipient after a DO-prime, the theme after a PO-prime), for two critical time windows during target sentence processing: the verb and the first syllable of the first postverbal noun (which was identical in theme and recipient). There was priming in both time windows, even though the verb differed between prime and target. Importantly, there was an inverse preference effect (i.e., stronger priming after a DO prime with a PO-biased verb than with a DO-biased verb) in the second time window. These results provide evidence for an error-based structure prediction system in comprehension. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Drifting pitch awareness after exposure to altered auditory feedback.

Various studies have claimed that the sense of agency is based on a comparison between an internal estimate of an action's outcome and sensory feedback. With respect to speech, this presumes that speakers have a stable prearticulatory representation of their own speech. However, recent research suggests that the sense of agency is flexible and thus in some contexts we may feel like we produced speech that was not actually produced by us. The current study tested whether the estimated pitch of one's articulation (termed pitch awareness) is affected by manipulated auditory feedback. In four experiments, 56 participants produced isolated vowels while being exposed to pitch-shifted auditory feedback. After every vocalization, participants indicated whether they thought the feedback was higher or lower than their actual production. After exposure to a block of high-pitched auditory feedback (+500 cents pitch shift), participants were more likely to label subsequent auditory feedback as "lower than my actual production," suggesting that prolonged exposure to high-pitched auditory feedback led to a drift in participants' pitch awareness. The opposite pattern was found after exposure to a constant -500 cents pitch shift. This suggests that pitch awareness is not solely based on a prearticulatory representation of intended speech or on a sensory prediction, but also on sensory feedback. We propose that this drift in pitch awareness could be indicative of a sense of agency over the pitch-shifted auditory feedback in the exposure block. If so, this suggests that the sense of agency over vocal output is flexible.

How do speakers resist distraction? Evidence from a taboo picture-word interference task.

Even in the presence of irrelevant stimuli, word production is a highly accurate and fluent process. But how do speakers prevent themselves from naming the wrong things? One possibility is that an attentional system inhibits task-irrelevant representations. Alternatively, a verbal self-monitoring system might check speech for accuracy and remove errors stemming from irrelevant information. Because self-monitoring is sensitive to social appropriateness, taboo errors should be intercepted more than neutral errors are. To prevent embarrassment, speakers might also speak more slowly when confronted with taboo distractors. Our results from two experiments are consistent with the self-monitoring account: Examining picture-naming speed (Experiment 1) and accuracy (Experiment 2), we found fewer naming errors but longer picture-naming latencies for pictures presented with taboo distractors than for pictures presented with neutral distractors. These results suggest that when intrusions of irrelevant words are highly undesirable, speakers do not simply inhibit these words: Rather, the language-production system adjusts itself to the context and filters out the undesirable words.

Are foreign-accented speakers that 'incredible'? The impact of the speaker's indexical properties on sentence processing.

This study investigated the impact of the speaker's identity generated by the voice on sentence processing. We examined the relation between ERP components associated with the processing of the voice (N100 and P200) from voice onset and those associated with sentence processing (N400 and late positivity) from critical word onset. We presented Dutch native speakers with sentences containing true (and known) information, unknown (but true) information or information violating world knowledge and had them perform a truth evaluation task. Sentences were spoken either in a native or a foreign accent. Truth evaluation judgments were not different for statements spoken by the native-accented and the foreign-accented speakers. Reduced N100 and P200 were observed in response to the foreign speaker's voice compared to the native speaker's. While statements containing unknown information or world knowledge violations generated a larger N400 than true statements in the native condition, they were not significantly different in the foreign condition, suggesting shallower processing of foreign-accented speech. The N100 was a significant predictor for the N400 in that the reduced N100 observed for the foreign speaker compared to the native speaker was related to a smaller N400 effect. These finding suggest that the impression of the speaker that listeners rapidly form from the voice affects semantic processing, which confirms that speaker's identity and language comprehension cannot be dissociated.

On the limits of shared syntactic representations: When word order variation blocks priming between an artificial language and Dutch.

Several studies used artificial language (AL) learning paradigms to investigate structural priming between languages in early phases of learning. The presence of such priming would indicate that these languages share syntactic representations. Muylle et al. (2020a) found similar priming of transitives and ditransitives between Dutch (SVO order) and an AL with either SVO or SOV order. However, it is unclear whether such sharing would occur if the AL allows both the same and different word order as the L1. Indeed, the presence of a (easy to share) similar structure might block (i.e., impede) sharing of a less similar structure. Here, we report 2 experiments that each tested 48 Dutch native speakers on an AL that allowed both SVO and SOV order in transitive and ditransitive sentences. We assessed both within-AL and AL-Dutch priming. We predicted (a) priming of both structure and word order within the AL, and (b) weaker AL-Dutch priming from SOV versus SVO sentences due to the presence of SVO sentences in the AL. Indeed, cross-linguistic priming was significantly weaker in SOV versus SVO conditions, but the blocking hypothesis was only supported by the transitive results. Unexpectedly, in the absence of a condition with verb overlap between prime and target sentences, no priming was found in AL and Dutch target conditions without verb overlap (Experiment 1), but priming emerged when a verb overlap condition was added (Experiment 2). This finding suggests that lexical repetition across sentences is crucial to establish abstract syntactic representations during early L2 acquisition. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Speaking with an alien voice: Flexible sense of agency during vocal production.

Speakers monitor auditory feedback during speech production in order to correct for speech errors. The comparator model proposes that this process is supported by comparing sensory feedback to internal predictions of the sensory consequences of articulation. Additionally, this comparison process is proposed to support the sense of agency over vocal output. The current study tests this hypothesis by asking whether mismatching auditory feedback leads to a decrease in the sense of agency as measured by speakers' responses to pitch-shifted feedback. Participants vocalized while auditory feedback was unexpectedly and briefly pitch-shifted. In addition, in one block, the entire vocalization's pitch was baseline-shifted ("alien voice"), while it was not in the other block ("normal voice"). Participants compensated for the pitch shifts even in the alien voice condition, suggesting that agency was flexible. This is problematic for the classic comparator model, where a mismatching feedback would lead to a loss of agency. Alternative models are discussed in light of these findings, including an adapted comparator model and the inferential account, which suggests that agency is inferred from the joint contribution of several multisensory sources of evidence. Together, these findings suggest that internal representations of one's own voice are more flexible than often assumed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Is a FAN always FUN? Phonological and orthographic effects in bilingual visual word recognition.

A visual semantic categorization task in English was performed by native English speakers (Experiment 1) and late bilinguals whose first language was Japanese (Experiment 2) or Spanish (Experiment 3). In the critical conditions, the target word was a homophone of a correct category exemplar (e.g., A BODY OF WATER--SEE; cf. SEA) or a word that differed from the correct exemplar by a phonological contrast absent in the bilinguals' first language (e.g., USED FOR COOLING DOWN--FUN; cf. FAN). Homophones elicited more false positive errors and slower processing than spelling controls in all groups. The Japanese-English bilinguals, but not the Spanish-English bilinguals, also displayed 'near-homophone' effects (i.e., homophone-like effects from minimal pairs on nonnative contrasts). We conclude that second-language visual word recognition is influenced by first-language phonology, although the effect is conditioned by the first-language orthographic system. Near-homophone effects can occur when the orthographic systems of the late bilingual's two languages are different in type (e.g., alphabetic vs. non-alphabetic), but may be blocked if the languages use the same writing script (e.g., Roman alphabet).