A Rose by Any Other Verb: The Effect of Expectations and Word Category on Processing Effort in Situated Sentence Comprehension.

Recent work has shown that linguistic and visual contexts jointly modulate linguistic expectancy and, thus, the processing effort for a (more or less) expected critical word. According to these findings, uncertainty about the upcoming referent in a visually-situated sentence can be reduced by exploiting the selectional restrictions of a preceding word (e.g., a verb or an adjective), which then reduces processing effort on the critical word (e.g., a referential noun). Interestingly, however, no such modulation was observed in these studies on the expectation-generating word itself. The goal of the current study is to investigate whether the reduction of uncertainty (i.e., the generation of expectations) simply does not modulate processing effort-or whether the particular subject-verb-object (SVO) sentence structure used in these studies (which emphasizes the referential nature of the noun as direct pointer to visually co-present objects) accounts for the observed pattern. To test these questions, the current design reverses the functional roles of nouns and verbs by using sentence constructions in which the noun reduces uncertainty about upcoming verbs, and the verb provides the disambiguating and reference-resolving piece of information. Experiment 1 (a Visual World Paradigm study) and Experiment 2 (a Grammaticality Maze study) both replicate the effect found in previous work (i.e., the effect of visually-situated context on the word which uniquely identifies the referent), albeit on the verb in the current study. Results on the noun, where uncertainty is reduced and expectations are generated in the current design, were mixed and were most likely influenced by design decisions specific to each experiment. These results show that processing of the reference-resolving word-whether it be a noun or a verb-reliably benefits from the prior linguistic and visual information that lead to the generation of concrete expectations.

Reevaluating pragmatic reasoning in language games.

The results of a highly influential study that tested the predictions of the Rational Speech Act (RSA) model suggest that (a) listeners use pragmatic reasoning in one-shot web-based referential communication games despite the artificial, highly constrained, and minimally interactive nature of the task, and (b) that RSA accurately captures this behavior. In this work, we reevaluate the contribution of the pragmatic reasoning formalized by RSA in explaining listener behavior by comparing RSA to a baseline literal listener model that is only driven by literal word meaning and the prior probability of referring to an object. Across three experiments we observe only modest evidence of pragmatic behavior in one-shot web-based language games, and only under very limited circumstances. We find that although RSA provides a strong fit to listener responses, it does not perform better than the baseline literal listener model. Our results suggest that while participants playing the role of the Speaker are informative in these one-shot web-based reference games, participants playing the role of the Listener only rarely take this Speaker behavior into account to reason about the intended referent. In addition, we show that RSA's fit is primarily due to a combination of non-pragmatic factors, perhaps the most surprising of which is that in the majority of conditions that are amenable to pragmatic reasoning, RSA (accurately) predicts that listeners will behave non-pragmatically. This leads us to conclude that RSA's strong overall correlation with human behavior in one-shot web-based language games does not reflect listener's pragmatic reasoning about informative speakers.

fMRI and acoustic analyses reveal neural correlates of gestural complexity and articulatory effort within bilateral inferior frontal gyrus during speech production.

In an event-related fMRI study of overt speech production, we investigated the relationship between gestural complexity and underlying brain activity within bilateral inferior frontal gyrus (IFG). We operationalized gestural complexity as the number of active articulatory tiers (glottal, oral, nasal) and the degree of fine-grained temporal coordination between tiers (low, high). Forty-three neurotypical participants produced three types of highly-frequent non-word CV-syllable sequences, which differ systematically in gestural complexity (simple: ['dadada], intermediate: ['tatata], complex: ['nanana]). Comparing blood oxygen level-dependent (BOLD) responses across complexity conditions revealed that syllables with greater gestural complexity elicited increased activation patterns. Moreover, when durational parameters were included as covariates in the analyses, significant effects of articulatory effort were found over and above the effects of complexity. The results suggest that these differences in BOLD-response reflect the differential contribution of articulatory mechanisms that are required to produce phonologically distinct speech sounds.

Social Context Modulates Tolerance for Pragmatic Violations in Binary but Not Graded Judgments.

A common method for investigating pragmatic processing and its development in children is to have participants make binary judgments of underinformative (UI) statements such as Some elephants are mammals. Rejection of such statements indicates that a (not-all) scalar implicature has been computed. Acceptance of UI statements is typically taken as evidence that the perceiver has not computed an implicature. Under this assumption, the results of binary judgment studies in children and adults suggest that computing an implicature may be cognitively costly. For instance, children under 7 years of age are systematically more likely to accept UI statements compared to adults. This makes sense if children have fewer processing resources than adults. However, Katsos and Bishop (2011) found that young children are able to detect violations of informativeness when given graded rather than binary response options. They propose that children simply have a greater tolerance for pragmatic violations than do adults. The present work examines whether this pragmatic tolerance plays a role in adult binary judgment tasks. We manipulated social attributes of a speaker in an attempt to influence how accepting a perceiver might be of the speaker's utterances. This manipulation affected acceptability rates for binary judgments (Experiment 1) but not for graded judgments (Experiment 2). These results raise concerns about the widespread use of binary choice tasks for investigating pragmatic processing and undermine the existing evidence suggesting that computing scalar implicatures is costly.

What do you know? ERP evidence for immediate use of common ground during online reference resolution.

Recent evidence on the time-course of conversational perspective taking is mixed. Some results suggest that listeners rapidly incorporate an interlocutor's knowledge during comprehension, while other findings suggest that listeners initially interpret language egocentrically. A key finding in support of the egocentric view comes from visual-world eye-tracking studies - listeners systematically look at potential referents that are known to them but unknown to the speaker. An alternative explanation is that eye movements might be driven by attentional processes that are unrelated to referent identification. To address this question, we assessed the time-course of perspective taking using event-related potentials (ERP). Participants were instructed to select a referent from a display of four animals (e.g., "Click on the brontosaurus with the boots") by a speaker who could only see three of the animals. A competitor (e.g., a brontosaurus with a purse) was either mutually visible, visible only to the listener, or absent from the display. Results showed that only the mutually visible competitor elicited an ERP signature of referential ambiguity. Critically, ERPs exhibited no evidence of referential confusion when the listener had privileged access to the competitor. Contra the egocentric hypothesis, this pattern of results indicates that listeners did not consider privileged competitors to be candidates for reference. These findings are consistent with theories of language processing that allow socio-pragmatic information to rapidly influence online language comprehension. The results also suggest that eye-tracking evidence in studies of online reference resolution may include distraction effects driven by privileged competitors and highlight the importance of using multiple measures to investigate perspective use.

Are subjective ratings of metaphors a red herring? The big two dimensions of metaphoric sentences.

What makes some metaphors easier to understand than others? Several psycholinguistic dimensions have been identified as candidate answers to this question, including appeals to familiarity and aptness. One way to operationalize these dimensions is to collect ratings of them from naive participants. In this article, we question the construct validity of this approach. Do ratings of aptness actually reflect the aptness of the metaphors? Are ratings of aptness measuring something different from ratings of familiarity? With two experiments and an analysis of existing datasets, we argue that ratings of metaphoric sentences are confounded by how easily people are able to understand the sentences (processing fluency). In the experiments, a context manipulation was designed to affect how fluently people would process the metaphors. Experiment 1 confirmed that the manipulation affected how quickly people understood the sentences in a response time task. Experiment 2 revealed that the same manipulation influenced ratings of such dimensions as familiarity and aptness. Finally, factor analyses-on the ratings data from Experiment 2 and from several existing datasets-revealed two underlying sources of variance in sentence-level ratings of metaphors (the "big two" dimensions of metaphoric sentences): processing fluency and figurativeness. We discuss the implications of these findings for theories of figurative-language processing by emphasizing more careful treatment of subjective ratings of metaphoric sentences, and by suggesting the use of alternative methods to manipulate and measure such dimensions as familiarity and aptness.

Conflict and surrender during sentence processing: an ERP study of syntax-semantics interaction.

Recent ERP studies report that implausible verb-argument combinations can elicit a centro-parietal P600 effect (e.g., "The hearty meal was devouring …"; Kim & Osterhout, 2005). Such eliciting conditions do not involve outright syntactic anomaly, deviating from previous reports of P600. Kim and Osterhout (2005) attributed such P600 effects to structural reprocessing that occurs when syntactic cues fail to support a semantically attractive interpretation ('meal' as the Agent of 'devouring') and the syntactic cues are overwhelmed; the sentence is therefore perceived as syntactically ill-formed. The current study replicated such findings and also found that altering the syntactic cues in such situations of syntax-semantics conflict (e.g., "The hearty meal would devour …") affects the conflict's outcome. P600s were eliminated when sentences contained syntactic cues that required multiple morphosyntactic steps to "repair". These sentences elicited a broad, left-anterior negativity at 300-600ms (LAN). We interpret the reduction in P600 amplitude in terms of "resistance" of syntactic cues to reprocessing. We speculate that the LAN may be generated by difficulty retrieving an analysis that satisfies both syntactic and semantic cues, which results when syntactic cues are strong enough to resist opposing semantic cues. This pattern of effects is consistent with partially independent but highly interactive syntactic and semantic processing streams, which often operate collaboratively but can compete for influence over interpretation.