Neural signatures of syntactic variation in speech planning.

Planning to speak is a challenge for the brain, and the challenge varies between and within languages. Yet, little is known about how neural processes react to these variable challenges beyond the planning of individual words. Here, we examine how fundamental differences in syntax shape the time course of sentence planning. Most languages treat alike (i.e., align with each other) the 2 uses of a word like "gardener" in "the gardener crouched" and in "the gardener planted trees." A minority keeps these formally distinct by adding special marking in 1 case, and some languages display both aligned and nonaligned expressions. Exploiting such a contrast in Hindi, we used electroencephalography (EEG) and eye tracking to suggest that this difference is associated with distinct patterns of neural processing and gaze behavior during early planning stages, preceding phonological word form preparation. Planning sentences with aligned expressions induces larger synchronization in the theta frequency band, suggesting higher working memory engagement, and more visual attention to agents than planning nonaligned sentences, suggesting delayed commitment to the relational details of the event. Furthermore, plain, unmarked expressions are associated with larger desynchronization in the alpha band than expressions with special markers, suggesting more engagement in information processing to keep overlapping structures distinct during planning. Our findings contrast with the observation that the form of aligned expressions is simpler, and they suggest that the global preference for alignment is driven not by its neurophysiological effect on sentence planning but by other sources, possibly by aspects of production flexibility and fluency or by sentence comprehension. This challenges current theories on how production and comprehension may affect the evolution and distribution of syntactic variants in the world's languages.

Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle.

Language models based on artificial neural networks increasingly capture key aspects of how humans process sentences. Most notably, model-based surprisals predict event-related potentials such as N400 amplitudes during parsing. Assuming that these models represent realistic estimates of human linguistic experience, their success in modeling language processing raises the possibility that the human processing system relies on no other principles than the general architecture of language models and on sufficient linguistic input. Here, we test this hypothesis on N400 effects observed during the processing of verb-final sentences in German, Basque, and Hindi. By stacking Bayesian generalised additive models, we show that, in each language, N400 amplitudes and topographies in the region of the verb are best predicted when model-based surprisals are complemented by an Agent Preference principle that transiently interprets initial role-ambiguous noun phrases as agents, leading to reanalysis when this interpretation fails. Our findings demonstrate the need for this principle independently of usage frequencies and structural differences between languages. The principle has an unequal force, however. Compared to surprisal, its effect is weakest in German, stronger in Hindi, and still stronger in Basque. This gradient is correlated with the extent to which grammars allow unmarked NPs to be patients, a structural feature that boosts reanalysis effects. We conclude that language models gain more neurobiological plausibility by incorporating an Agent Preference. Conversely, theories of human processing profit from incorporating surprisal estimates in addition to principles like the Agent Preference, which arguably have distinct evolutionary roots.

An Agent-First Preference in a Patient-First Language During Sentence Comprehension.

The language comprehension system preferentially assumes that agents come first during incremental processing. While this might reflect a biologically fixed bias, shared with other domains and other species, the evidence is limited to languages that place agents first, and so the bias could also be learned from usage frequency. Here, we probe the bias with electroencephalography (EEG) in Äiwoo, a language that by default places patients first, but where sentence-initial nouns are still locally ambiguous between patient or agent roles. Comprehenders transiently interpreted nonhuman nouns as patients, eliciting a negativity when disambiguation was toward the less common agent-initial order. By contrast and against frequencies, human nouns were transiently interpreted as agents, eliciting an N400-like negativity when the disambiguation was toward patient-initial order. Consistent with the notion of a fixed property, the agent bias is robust against usage frequency for human referents. However, this bias can be reversed by frequency experience for nonhuman referents.

The Agent Preference in Visual Event Apprehension.

A central aspect of human experience and communication is understanding events in terms of agent ("doer") and patient ("undergoer" of action) roles. These event roles are rooted in general cognition and prominently encoded in language, with agents appearing as more salient and preferred over patients. An unresolved question is whether this preference for agents already operates during apprehension, that is, the earliest stage of event processing, and if so, whether the effect persists across different animacy configurations and task demands. Here we contrast event apprehension in two tasks and two languages that encode agents differently; Basque, a language that explicitly case-marks agents ('ergative'), and Spanish, which does not mark agents. In two brief exposure experiments, native Basque and Spanish speakers saw pictures for only 300 ms, and subsequently described them or answered probe questions about them. We compared eye fixations and behavioral correlates of event role extraction with Bayesian regression. Agents received more attention and were recognized better across languages and tasks. At the same time, language and task demands affected the attention to agents. Our findings show that a general preference for agents exists in event apprehension, but it can be modulated by task and language demands.

Cross-linguistic differences in case marking shape neural power dynamics and gaze behavior during sentence planning.

Languages differ in how they mark the dependencies between verbs and arguments, e.g., by case. An eye tracking and EEG picture description study examined the influence of case marking on the time course of sentence planning in Basque and Swiss German. While German assigns an unmarked (nominative) case to subjects, Basque specifically marks agent arguments through ergative case. Fixations to agents and event-related synchronization (ERS) in the theta and alpha frequency bands, as well as desynchronization (ERD) in the alpha and beta bands revealed multiple effects of case marking on the time course of early sentence planning. Speakers decided on case marking under planning early when preparing sentences with ergative-marked agents in Basque, whereas sentences with unmarked agents allowed delaying structural commitment across languages. These findings support hierarchically incremental accounts of sentence planning and highlight how cross-linguistic differences shape the neural dynamics underpinning language use.

Speaking for seeing: Sentence structure guides visual event apprehension.

Human experience and communication are centred on events, and event apprehension is a rapid process that draws on the visual perception and immediate categorization of event roles ("who does what to whom"). We demonstrate a role for syntactic structure in visual information uptake for event apprehension. An event structure foregrounding either the agent or patient was activated during speaking, transiently modulating the apprehension of subsequently viewed unrelated events. Speakers of Dutch described pictures with actives and passives (agent and patient foregrounding, respectively). First fixations on pictures of unrelated events that were briefly presented (for 300 ms) next were influenced by the active or passive structure of the previously produced sentence. Going beyond the study of how single words cue object perception, we show that sentence structure guides the viewpoint taken during rapid event apprehension.

Speech Planning at Turn Transitions in Dialog Is Associated With Increased Processing Load.

Speech planning is a sophisticated process. In dialog, it regularly starts in overlap with an incoming turn by a conversation partner. We show that planning spoken responses in overlap with incoming turns is associated with higher processing load than planning in silence. In a dialogic experiment, participants took turns with a confederate describing lists of objects. The confederate's utterances (to which participants responded) were pre-recorded and varied in whether they ended in a verb or an object noun and whether this ending was predictable or not. We found that response planning in overlap with sentence-final verbs evokes larger task-evoked pupillary responses, while end predictability had no effect. This finding indicates that planning in overlap leads to higher processing load for next speakers in dialog and that next speakers do not proactively modulate the time course of their response planning based on their predictions of turn endings. The turn-taking system exerts pressure on the language processing system by pushing speakers to plan in overlap despite the ensuing increase in processing load.

Motion vector field phase-to-amplitude resampling for 4D motion-compensated cone-beam CT.

We propose a phase-to-amplitude resampling (PTAR) method to reduce motion blurring in motion-compensated (MoCo) 4D cone-beam CT (CBCT) image reconstruction, without increasing the computational complexity of the motion vector field (MVF) estimation approach. PTAR is able to improve the image quality in reconstructed 4D volumes, including both regular and irregular respiration patterns. The PTAR approach starts with a robust phase-gating procedure for the initial MVF estimation and then switches to a phase-adapted amplitude gating method. The switch implies an MVF-resampling, which makes them amplitude-specific. PTAR ensures that the MVFs, which have been estimated on phase-gated reconstructions, are still valid for all amplitude-gated reconstructions. To validate the method, we use an artificially deformed clinical CT scan with a realistic breathing pattern and several patient data sets acquired with a TrueBeamTM integrated imaging system (Varian Medical Systems, Palo Alto, CA, USA). Motion blurring, which still occurs around the area of the diaphragm or at small vessels above the diaphragm in artifact-specific cyclic motion compensation (acMoCo) images based on phase-gating, is significantly reduced by PTAR. Also, small lung structures appear sharper in the images. This is demonstrated both for simulated and real patient data. A quantification of the sharpness of the diaphragm confirms these findings. PTAR improves the image quality of 4D MoCo reconstructions compared to conventional phase-gated MoCo images, in particular for irregular breathing patterns. Thus, PTAR increases the robustness of MoCo reconstructions for CBCT. Because PTAR does not require any additional steps for the MVF estimation, it is computationally efficient. Our method is not restricted to CBCT but could rather be applied to other image modalities.

Two methods for reducing moving metal artifacts in cone-beam CT.

PURPOSE: In image-guided radiation therapy, fiducial markers or clips are often used to determine the position of the tumor. These markers lead to streak artifacts in cone-beam CT (CBCT) scans. Standard inpainting-based metal artifact reduction (MAR) methods fail to remove these artifacts in cases of large motion. We propose two methods to effectively reduce artifacts caused by moving metal inserts. METHODS: The first method (MMAR) utilizes a coarse metal segmentation in the image domain and a refined segmentation in the rawdata domain. After an initial reconstruction, metal is segmented and forward projected giving a coarse metal mask in the rawdata domain. Inside the coarse mask, metal is segmented by utilizing a 2D Sobel filter. Metal is removed by linear interpolation in the refined metal mask. The second method (MoCoMAR) utilizes a motion compensation (MoCo) algorithm [Med Phys. 2013;40:101913] that provides us with a motion-free volume (3D) or with a time series of motion-free volumes (4D). We then apply the normalized metal artifact reduction (NMAR) [Med Phys. 2010;37:5482-5493] to these MoCo volumes. Both methods were applied to three CBCT data sets of patients with metal inserts in the thorax or abdomen region and a 4D thorax simulation. The results were compared to volumes corrected by a standard MAR1 [Radiology. 1987;164:576-577]. RESULTS: MMAR and MoCoMAR were able to remove all artifacts caused by moving metal inserts for the patients and the simulation. Both new methods outperformed the standard MAR1, which was only able to remove artifacts caused by metal inserts with little or no motion. CONCLUSIONS: In this work, two new methods to remove artifacts caused by moving metal inserts are introduced. Both methods showed good results for a simulation and three patients. While the first method (MMAR) works without any prior knowledge, the second method (MoCoMAR) requires a respiratory signal for the MoCo step and is computationally more demanding and gives no benefit over MMAR, unless MoCo images are desired.

Word Order and Voice Influence the Timing of Verb Planning in German Sentence Production.

Theories of incremental sentence production make different assumptions about when speakers encode information about described events and when verbs are selected, accordingly. An eye tracking experiment on German testing the predictions from linear and hierarchical incrementality about the timing of event encoding and verb planning is reported. In the experiment, participants described depictions of two-participant events with sentences that differed in voice and word order. Verb-medial active sentences and actives and passives with sentence-final verbs were compared. Linear incrementality predicts that sentences with verbs placed early differ from verb-final sentences because verbs are assumed to only be planned shortly before they are articulated. By contrast, hierarchical incrementality assumes that speakers start planning with relational encoding of the event. A weak version of hierarchical incrementality assumes that only the action is encoded at the outset of formulation and selection of lexical verbs only occurs shortly before they are articulated, leading to the prediction of different fixation patterns for verb-medial and verb-final sentences. A strong version of hierarchical incrementality predicts no differences between verb-medial and verb-final sentences because it assumes that verbs are always lexically selected early in the formulation process. Based on growth curve analyses of fixations to agent and patient characters in the described pictures, and the influence of character humanness and the lack of an influence of the visual salience of characters on speakers' choice of active or passive voice, the current results suggest that while verb planning does not necessarily occur early during formulation, speakers of German always create an event representation early.

Verbal Semantics Drives Early Anticipatory Eye Movements during the Comprehension of Verb-Initial Sentences.

Studies on anticipatory processes during sentence comprehension often focus on the prediction of postverbal direct objects. In subject-initial languages (the target of most studies so far), however, the position in the sentence, the syntactic function, and the semantic role of arguments are often conflated. For example, in the sentence "The frog will eat the fly" the syntactic object ("fly") is at the same time also the last word and the patient argument of the verb. It is therefore not apparent which kind of information listeners orient to for predictive processing during sentence comprehension. A visual world eye tracking study on the verb-initial language Tagalog (Austronesian) tested what kind of information listeners use to anticipate upcoming postverbal linguistic input. The grammatical structure of Tagalog allows to test whether listeners' anticipatory gaze behavior is guided by predictions of the linear order of words, by syntactic functions (e.g., subject/object), or by semantic roles (agent/patient). Participants heard sentences of the type "Eat frog fly" or "Eat fly frog" (both meaning "The frog will eat the fly") while looking at displays containing an agent referent ("frog"), a patient referent ("fly") and a distractor. The verb carried morphological marking that allowed the order and syntactic function of agent and patient to be inferred. After having heard the verb, listeners fixated on the agent irrespective of its syntactic function or position in the sentence. While hearing the first-mentioned argument, listeners fixated on the corresponding referent in the display accordingly and then initiated saccades to the last-mentioned referent before it was encountered. The results indicate that listeners used verbal semantics to identify referents and their semantic roles early; information about word order or syntactic functions did not influence anticipatory gaze behavior directly after the verb was heard. In this verb-initial language, event semantics takes early precedence during the comprehension of sentences, while arguments are anticipated temporally more local to when they are encountered. The current experiment thus helps to better understand anticipation during language processing by employing linguistic structures not available in previously studied subject-initial languages.

The Timing of Utterance Planning in Task-Oriented Dialogue: Evidence from a Novel List-Completion Paradigm.

In conversation, interlocutors rarely leave long gaps between turns, suggesting that next speakers begin to plan their turns while listening to the previous speaker. The present experiment used analyses of speech onset latencies and eye-movements in a task-oriented dialogue paradigm to investigate when speakers start planning their responses. German speakers heard a confederate describe sets of objects in utterances that either ended in a noun [e.g., Ich habe eine Tür und ein Fahrrad ("I have a door and a bicycle")] or a verb form [e.g., Ich habe eine Tür und ein Fahrrad besorgt ("I have gotten a door and a bicycle")], while the presence or absence of the final verb either was or was not predictable from the preceding sentence structure. In response, participants had to name any unnamed objects they could see in their own displays with utterances such as Ich habe ein Ei ("I have an egg"). The results show that speakers begin to plan their turns as soon as sufficient information is available to do so, irrespective of further incoming words.

Cardiorespiratory motion-compensated micro-CT image reconstruction using an artifact model-based motion estimation.

PURPOSE: Cardiac in vivo micro-CT imaging of small animals typically requires double gating due to long scan times and high respiratory rates. The simultaneous respiratory and cardiac gating can either be done prospectively or retrospectively. In any case, for true 5D imaging, i.e., three spatial dimensions plus one respiratory-temporal dimension plus one cardiac temporal dimension, the amount of information corresponding to a given respiratory and cardiac phase is orders of magnitude lower than the total amount of information acquired. Achieving similar image quality for 5D than for usual 3D investigations would require increasing the amount of data and thus the applied dose to the animal. Therefore, the goal is phase-correlated imaging with high image quality but without increasing the dose level. METHODS: To achieve this, the authors propose a new image reconstruction algorithm that makes use of all available projection data, also of that corresponding to other motion windows. In particular, the authors apply a motion-compensated image reconstruction approach that sequentially compensates for respiratory and cardiac motion to decrease the impact of sparsification. In that process, all projection data are used no matter which motion phase they were acquired in. Respiratory and cardiac motion are compensated for by using motion vector fields. These motion vector fields are estimated from initial phase-correlated reconstructions based on a deformable registration approach. To decrease the sensitivity of the registration to sparse-view artifacts, an artifact model-based approach is used including a cyclic consistent nonrigid registration algorithm. RESULTS: The preliminary results indicate that the authors' approach removes the sparse-view artifacts of conventional phase-correlated reconstructions while maintaining temporal resolution. In addition, it achieves noise levels and spatial resolution comparable to that of nongated reconstructions due to the improved dose usage. By using the proposed motion estimation, no sensitivity to streaking artifacts has been observed. CONCLUSIONS: Using sequential double gating combined with artifact model-based motion estimation allows to accurately estimate respiratory and cardiac motion from highly undersampled data. No sensitivity to streaking artifacts introduced by sparse angular sampling has been observed for the investigated dose levels. The motion-compensated image reconstruction was able to correct for both, respiratory and cardiac motion, by applying the estimated motion vector fields. The administered dose per animal can thus be reduced for 5D imaging allowing for longitudinal studies at the highest image quality.