Musical coordination affects children's perspective-taking, but musical synchrony does not.

Perspective-taking, which is important for communication and social activities, can be cultivated through joint actions, including musical activities in children. We examined how rhythmic activities requiring coordination affect perspective-taking in a referential communication task with 100 Chinese 4- to 6-year-old children. In Study 1, 5- to 6-year-old children played an instrument with a virtual partner in one of three coordination conditions: synchrony, asynchrony, and antiphase synchrony. Eye movements were then monitored with the partner giving instructions to identify a shape referent which included a pre-nominal scalar adjective (e.g., big cubic block). When the target contrast (a small cubic block) was in the shared ground and a competitor contrast was occluded for the partner, participants who used perspective differences could, in principle, identify the intended referent before the shape was named. We hypothesized that asynchronous and antiphase synchronous musical activities, which require self-other distinction, might have stronger effects on perspective-taking than synchronous activity. Children in the asynchrony and antiphase synchrony conditions, but not the synchrony condition, showed anticipatory looks at the target, demonstrating real-time use of the partner's perspective. Study 2 was conducted to determine if asynchrony and antiphase asynchrony resulted in perspective-taking that otherwise would not have been observed, or if synchronous coordination inhibited perspective-taking that would otherwise have occurred. We found no evidence for online perspective-taking in 4- to 6-year-old children without music manipulation. Therefore, playing instruments asynchronously or in alternation, but not synchronously, increases perspective-taking in children of this age, likely by training self-other distinction and control. A video abstract of this article can be viewed at https://youtu.be/TM9h_GpFlsA. RESEARCH HIGHLIGHTS: This study is the first to show that rhythmic coordination, a form of non-linguistic interaction, can affect children's performance in a subsequent linguistic task. Eye-movement data revealed that children's perspective-taking in language processing was facilitated by prior asynchronous and antiphase synchronous musical interactions, but not by synchronous coordination. The results challenge the common "similar is better" view, suggesting that maintaining self-other distinction may benefit social interactions that involve representing individual differences.

Now or later: Perceptual data are not immediately forgotten during language processing.

Christiansen & Chater (C&C) propose that language comprehenders must immediately compress perceptual data by "chunking" them into higher-level categories. Effective language understanding, however, requires maintaining perceptual information long enough to integrate it with downstream cues. Indeed, recent results suggest comprehenders do this. Although cognitive systems are undoubtedly limited, frameworks that do not take into account the tasks that these systems evolved to solve risk missing important insights.

Inferring Difficulty: Flexibility in the Real-time Processing of Disfluency.

Upon hearing a disfluent referring expression, listeners expect the speaker to refer to an object that is previously unmentioned, an object that does not have a straightforward label, or an object that requires a longer description. Two visual-world eye-tracking experiments examined whether listeners directly associate disfluency with these properties of objects, or whether disfluency attribution is more flexible and involves situation-specific inferences. Since in natural situations reference to objects that do not have a straightforward label or that require a longer description is correlated with both production difficulty and with disfluency, we used a mini-artificial lexicon to dissociate difficulty from these properties, building on the fact that recently learned names take longer to produce than existing words in one's mental lexicon. The results demonstrate that disfluency attribution involves situation-specific inferences; we propose that in new situations listeners spontaneously infer what may cause production difficulty. However, the results show that these situation-specific inferences are limited in scope: listeners assessed difficulty relative to their own experience with the artificial names, and did not adapt to the assumed knowledge of the speaker.

Asking or Telling--Real-time Processing of Prosodically Distinguished Questions and Statements.

We introduce a targeted language game approach using the visual world, eye-movement paradigm to assess when and how certain intonational contours affect the interpretation of utterances. We created a computer-based card game in which elliptical utterances such as "Got a candy" occurred with a nuclear contour most consistent with a yes-no question (H* H-H%) or a statement (L* L-L%). In Experiment I we explored how such contours are integrated online. In Experiment 2 we studied the expectations listeners have for how intonational contours signal intentions: do these reflect linguistic categories or rapid adaptation to the paradigm? Prosody had an immediate effect on interpretation, as indexed by the pattern and timing of fixations. Moreover, the association between different contours and intentions was quite robust in the absence of clear syntactic cues to sentence type, and was not due to rapid adaptation. Prosody had immediate effects on interpretation even though there was a construction-based bias to interpret "got a" as a question. Taken together, we believe this paradigm will provide further insights into how intonational contours and their phonetic realization interact with other cues to sentence type in online comprehension.

Syntactic priming without lexical overlap in reading comprehension.

Syntactic priming without lexical overlap is well-documented in language production. In contrast, reading-time comprehension studies, which typically use locally ambiguous sentences, generally find syntactic priming only with lexical overlap. This asymmetry has led some researchers to propose that distinct mechanisms underlie the comprehension and production of syntactic structure. Instead, we propose that methodological differences in how priming is assessed are largely responsible for the asymmetry: in comprehension, lexical biases in a locally ambiguous target sentence may overwhelm the influence of syntactic priming effects on a reader's interpretation. We addressed these issues in a self-paced reading study by (1) using target sentences containing global attachment ambiguities, (2) examining a syntactic structure which does not involve an argument of the verb, and (3) factoring out the unavoidable lexical biases associated with the target sentences in a mixed-effects regression model. Under these conditions, syntactic priming affected how ambiguous sentences were parsed, and facilitated reading times when target sentences were parsed using the primed structure. This resolves discrepancies among previous findings, and suggests that the same mechanism underlies syntactic priming in comprehension and production.

Spontaneous perspective-taking in real-time language comprehension: evidence from eye-movements and grain of coordination.

Linguistic communication requires interlocutors to consider differences in each other's knowledge (perspective-taking). However, perspective-taking might either be spontaneous or strategic. We monitored listeners' eye movements in a referential communication task. A virtual speaker gave temporally ambiguous instructions with scalar adjectives ("big" in "big cubic block"). Scalar adjectives assume a contrasting object (a small cubic block). We manipulated whether the contrasting object (a small triangle) for a competitor object (a big triangle) was in common ground (visible to both speaker and listener) or was occluded so it was in the listener's privileged ground, in which case perspective-taking would allow earlier reference-resolution. We used a complex visual context with multiple objects, making strategic perspective-taking unlikely when all objects are in the listener's referential domain. A turn-taking, puzzle-solving task manipulated whether participants could anticipate a more restricted referential domain. Pieces were either confined to a small area (requiring fine-grained coordination) or distributed across spatially distinct regions (requiring only coarse-grained coordination). Results strongly supported spontaneous perspective-taking: Although comprehension was less time-locked in the coarse-grained condition, participants in both conditions used perspective information to identify the target referent earlier when the competitor contrast was in privileged ground, even when participants believed instructions were computer-generated.

Event-related brain potentials in lexical processing with Chinese characters show effects of contextual diversity but not word frequency.

The diversity of contexts in which a word occurs, operationalized as CD, is strongly correlated with response times in visual word recognition, with higher CD words being recognized faster. CD and token word frequency (WF) are highly correlated but in behavioral studies when other variables that affect word visual recognition are controlled for, the WF effect is eliminated when contextual diversity (CD) is controlled. In contrast, the only event-related potential (ERP) study to examine CD and WF Vergara-Martínez et al., Cognitive, Affective, & Behavioral Neuroscience, 17, 461-474, (2017) found effects of both WF and CD with different distributions in the 225- to 325-ms time window. We conducted an ERP study with Chinese characters to explore the neurocognitive dynamics of WF and CD. We compared three groups of characters: (1) characters high in frequency and low in CD; (2) characters low in frequency and low in CD; and (3) characters high in frequency and high in CD. Behavioral data showed significant effects of CD but not WF. Character CD, but not character frequency, modulated the late positive component (LPC): high-CD characters elicited a larger LPC, widely distributed, with largest amplitude at the posterior sites compared to low-CD characters in the 400-to 600-ms time window, consistent with earlier ERP studies of WF in Chinese, and with the hypothesis that CD affects semantic and context-based processes. No WF effect on any ERP components was observed when CD was controlled. The results are consistent with behavioral results showing CD but not WF effects, and in particular with a "context constructionist" framework.

Prediction, explanation, and the role of generative models in language processing.

We propose, following Clark, that generative models also play a central role in the perception and interpretation of linguistic signals. The data explanation approach provides a rationale for the role of prediction in language processing and unifies a number of phenomena, including multiple-cue integration, adaptation effects, and cortical responses to violations of linguistic expectations.

Neural correlates of partial lexical activation.

As a spoken word unfolds over time, it is temporarily consistent with the acoustic forms of multiple words. Previous behavioral research has shown that, in the face of temporary ambiguity about how a word will end, multiple candidate words are briefly activated. Here, we provide neural imaging evidence that lexical candidates only temporarily consistent with the input activate perceptually based semantic representations. An artificial lexicon and novel visual environment were used to target human MT/V5 and an area anterior to it which have been shown to be recruited during the reading of motion words. Participants learned words that referred to novel objects and to motion or color/texture changes that the objects underwent. The lexical items corresponding to the change events were organized into phonologically similar pairs differing only in the final syllable. Upon hearing spoken scene descriptions in a posttraining verification task, participants showed greater activation in the left hemisphere anterior extent of MT/V5 when motion words were heard than when nonmotion words were heard. Importantly, when a nonmotion word was heard, the level of activation in the anterior extent of MT/V5 was modulated by whether there was a phonologically related competitor that was a motion word rather than another nonmotion word. These results provide evidence of activation of a perceptual brain region in response to the semantics of a word while lexical competition is in process and before the word is fully recognized.

Syllable Inference as a Mechanism for Spoken Language Understanding.

A classic problem in spoken language comprehension is how listeners perceive speech as being composed of discrete words, given the variable time-course of information in continuous signals. We propose a syllable inference account of spoken word recognition and segmentation, according to which alternative hierarchical models of syllables, words, and phonemes are dynamically posited, which are expected to maximally predict incoming sensory input. Generative models are combined with current estimates of context speech rate drawn from neural oscillatory dynamics, which are sensitive to amplitude rises. Over time, models which result in local minima in error between predicted and recently experienced signals give rise to perceptions of hearing words. Three experiments using the visual world eye-tracking paradigm with a picture-selection task tested hypotheses motivated by this framework. Materials were sentences that were acoustically ambiguous in numbers of syllables, words, and phonemes they contained (cf. English plural constructions, such as "saw (a) raccoon(s) swimming," which have two loci of grammatical information). Time-compressing, or expanding, speech materials permitted determination of how temporal information at, or in the context of, each locus affected looks to, and selection of, pictures with a singular or plural referent (e.g., one or more than one raccoon). Supporting our account, listeners probabilistically interpreted identical chunks of speech as consistent with a singular or plural referent to a degree that was based on the chunk's gradient rate in relation to its context. We interpret these results as evidence that arriving temporal information, judged in relation to language model predictions generated from context speech rate evaluated on a continuous scale, informs inferences about syllables, thereby giving rise to perceptual experiences of understanding spoken language as words separated in time.

Predictions of Miscommunication in Verbal Communication During Collaborative Joint Action.

Purpose The purpose of the current study was to examine the lexical and pragmatic factors that may contribute to turn-by-turn failures in communication (i.e., miscommunication) that arise regularly in interactive communication. Method Using a corpus from a collaborative dyadic building task, we investigated what differentiated successful from unsuccessful communication and potential factors associated with the choice to provide greater lexical information to a conversation partner. Results We found that more successful dyads' language tended to be associated with greater lexical density, lower ambiguity, and fewer questions. We also found participants were more lexically dense when accepting and integrating a partner's information (i.e., grounding) but were less lexically dense when responding to a question. Finally, an exploratory analysis suggested that dyads tended to spend more lexical effort when responding to an inquiry and used assent language accurately-that is, only when communication was successful. Conclusion Together, the results suggest that miscommunication both emerges and benefits from ambiguous and lexically dense utterances.

Effects of coordination and gender on prosocial behavior in 4-year-old Chinese children.

In a block-assembly task with 138, 4-year-old Chinese kindergarten children, tested in pairs, we manipulated whether fine-grained coordination was required for accomplishing a shared goal with the same end product: building two adjoined towers with alternating levels of orange and green colored blocks to match a depicted model. In the coordination condition, each child had blocks of only one color and built the towers together. In the shared-goal-only condition, each child had both color blocks and built one of the towers, which they then adjoined. We predicted that children in the coordination condition would be more prosocial than children in the shared-goal-only condition. Studies with Western children typically find that girls are more generous than boys. However, we predicted the opposite pattern because Chinese culture emphasizes the importance of generosity more for males than females. Children in the coordination condition were more willing to help their partner complete an unrelated task and were more generous in sharing stickers with unknown children in a dictator game. These results demonstrate that level of coordination affects prosociality above and beyond having a shared goal, and are the first demonstration that prosocial effects of a collaborative task with children generalize beyond the participants to anonymous strangers. Boys shared more stickers with unknown children than girls, suggesting that gender differences in generosity are, in part, culturally conditioned.

Phonological typicality and sentence processing.

In studies of language, it is widely accepted that the form of a word is independent of its meaning and syntactic category. Thus, the relationship between phonological form and grammatical class would not be expected to affect reading time. However, Farmer et al. have now shown that the phonological typicality of a noun or verb influences how rapidly it is read. This finding has implications for both sentence processing and the interpretation of fixation patterns in reading.

The role of perspective in identifying domains of reference.

We used the contrastive expectation associated with scalar adjectives to examine whether listeners are sensitive to the distinction between common and privileged information during real-time reference resolution. Our results show that listeners used this distinction to narrow the set of potential referents to objects with contrasts in common ground from the earliest moments. These results extend previous evidence that ground information influences real-time language processing by showing that the distinction between common and privileged information is used without being triggered by unusual circumstances.

Immediate effects of form-class constraints on spoken word recognition.

In many domains of cognitive processing there is strong support for bottom-up priority and delayed top-down (contextual) integration. We ask whether this applies to supra-lexical context that could potentially constrain lexical access. Previous findings of early context integration in word recognition have typically used constraints that can be linked to pair-wise conceptual relations between words. Using an artificial lexicon, we found immediate integration of syntactic expectations based on pragmatic constraints linked to syntactic categories rather than words: phonologically similar "nouns" and "adjectives" did not compete when a combination of syntactic and visual information strongly predicted form class. These results suggest that predictive context is integrated continuously, and that previous findings supporting delayed context integration stem from weak contexts rather than delayed integration.

Addressees distinguish shared from private information when interpreting questions during interactive conversation.

Two experiments examined the role of common ground in the production and on-line interpretation of wh-questions such as What's above the cow with shoes? Experiment 1 examined unscripted conversation, and found that speakers consistently use wh-questions to inquire about information known only to the addressee. Addressees were sensitive to this tendency, and quickly directed attention toward private entities when interpreting these questions. A second experiment replicated the interpretation findings in a more constrained setting. These results add to previous evidence that the common ground influences initial language processes, and suggests that the strength and polarity of common ground effects may depend on contributions of sentence type as well as the interactivity of the situation.

Heeding the voice of experience: the role of talker variation in lexical access.

Two experiments used the head-mounted eye-tracking methodology to examine the time course of lexical activation in the face of a non-phonemic cue, talker variation. We found that lexical competition was attenuated by consistent talker differences between words that would otherwise be lexical competitors. In Experiment 1, some English cohort word-pairs were consistently spoken by a single talker (male couch, male cows), while other word-pairs were spoken by different talkers (male sheep, female sheet). After repeated instances of talker-word pairings, words from different-talker pairs showed smaller proportions of competitor fixations than words from same-talker pairs. In Experiment 2, participants learned to identify black-and-white shapes from novel labels spoken by one of two talkers. All of the 16 novel labels were VCVCV word-forms atypical of, but not phonologically illegal in, English. Again, a word was consistently spoken by one talker, and its cohort or rhyme competitor was consistently spoken either by that same talker (same-talker competitor) or the other talker (different-talker competitor). Targets with different-talker cohorts received greater fixation proportions than targets with same-talker cohorts, while the reverse was true for fixations to cohort competitors; there were fewer erroneous selections of competitor referents for different-talker competitors than same-talker competitors. Overall, these results support a view of the lexicon in which entries contain extra-phonemic information. Extensions of the artificial lexicon paradigm and developmental implications are discussed.

Tic Tac Toe: effects of predictability and importance on acoustic prominence in language production.

Importance and predictability each have been argued to contribute to acoustic prominence. To investigate whether these factors are independent or two aspects of the same phenomenon, naïve participants played a verbal variant of Tic Tac Toe. Both importance and predictability contributed independently to the acoustic prominence of a word, but in different ways. Predictable game moves were shorter in duration and had less pitch excursion than less predictable game moves, whereas intensity was higher for important game moves. These data also suggest that acoustic prominence is affected by both speaker-centered processes (speaker effort) and listener-centered processes (intent to signal important information to the listener).

Perception of speech reflects optimal use of probabilistic speech cues.

Listeners are exquisitely sensitive to fine-grained acoustic detail within phonetic categories for sounds and words. Here we show that this sensitivity is optimal given the probabilistic nature of speech cues. We manipulated the probability distribution of one probabilistic cue, voice onset time (VOT), which differentiates word initial labial stops in English (e.g., "beach" and "peach"). Participants categorized words from distributions of VOT with wide or narrow variances. Uncertainty about word identity was measured by four-alternative forced-choice judgments and by the probability of looks to pictures. Both measures closely reflected the posterior probability of the word given the likelihood distributions of VOT, suggesting that listeners are sensitive to these distributions.

Gradient sensitivity to within-category variation in words and syllables.

Five experiments monitored eye movements in phoneme and lexical identification tasks to examine the effect of within-category subphonetic variation on the perception of stop consonants. Experiment 1 demonstrated gradient effects along voice-onset time (VOT) continua made from natural speech, replicating results with synthetic speech (B. McMurray, M. K. Tanenhaus, & R. N. Aslin, 2002). Experiments 2-5 used synthetic VOT continua to examine effects of response alternatives (2 vs. 4), task (lexical vs. phoneme decision), and type of token (word vs. consonant-vowel). A gradient effect of VOT in at least one half of the continuum was observed in all conditions. These results suggest that during online spoken word recognition, lexical competitors are activated in proportion to their continuous distance from a category boundary. This gradient processing may allow listeners to anticipate upcoming acoustic-phonetic information in the speech signal and dynamically compensate for acoustic variability.