Contrasting behavioral looking procedures: a case study on infant speech segmentation.

This paper compared three different procedures common in infant speech perception research: a headturn preference procedure (HPP) and a central-fixation (CF) procedure with either automated eye-tracking (CF-ET) or manual coding (CF-M). In theory, such procedures all measure the same underlying speech perception and learning mechanisms and the choice between them should ideally be irrelevant in unveiling infant preference. However, the ManyBabies study (ManyBabies Consortium, 2019), a cross-laboratory collaboration on infants' preference for child-directed speech, revealed that choice of procedure can modulate effect sizes. Here we examined whether procedure also modulates preference in paradigms that add a learning phase prior to test: a speech segmentation paradigm. Such paradigms are particularly important for studying the learning mechanisms infants can employ for language acquisition. We carried out the same familiarization-then-test experiment with the three different procedures (32 unique infants per procedure). Procedures were compared on various factors, such as overall effect, average looking time and drop-out rate. The key observations are that the HPP yielded a larger familiarity preference, but also reported larger drop-out rates. This raises questions about the generalizability of results. We argue that more collaborative research into different procedures in infant preference experiments is required in order to interpret the variation in infant preferences more accurately.

Cortical auditory responses index the contributions of different RMS-level-dependent segments to speech intelligibility.

Previous behavioral and neurophysiological studies indicated that the use of an appropriate segmentation method to parse speech streams into meaningful chunks is of vital importance for the examination of sentence perception and intelligibility. Researchers have recently proposed speech segmentation methods employing the relative root-mean-square (RMS) intensity to separate sentences into segments with distinct intelligibility information. However, the effects of different RMS-level segments containing distinct intelligibility information on neural oscillations are not clear. Using scalp-recorded electroencephalography (EEG) data, we investigated the hypothesis that perceptual responses to different RMS-level-dependent speech segments would have distinct EEG characteristics derived from the power values at each frequency band and the relationship between acoustics and neural oscillations at different response time and spatial distribution. We analyzed the EEG power and synchronized neural oscillations corresponding to auditory temporal fluctuations when subjects listened to Mandarin sentences with only high-RMS-level segments and only middle-RMS-level segments preserved, respectively. The results showed significantly stronger EEG spectral power in the delta and theta bands for high-RMS-level stimuli compared with middle-RMS-level stimuli, indicating that the former carry more speech-parsing information at the syllabic level. Differences in neural synchronization were also found between the high- and middle-RMS-level stimuli, allowing for the derivation of intelligibility indices for cortical responses corresponding to different RMS-level segments. These findings suggest that both high- and middle-RMS-level segments drive delta and theta rhythms to track stimuli, and that neural oscillations employ different tracking patterns for these two segment types during auditory sentence processing. Moreover, they suggest that neural oscillations can serve as effective indices for the identification of reliable intelligibility factors in RMS-level-dependent stimuli.

Statistical learning of speech regularities can occur outside the focus of attention.

Statistical learning, the process of extracting regularities from the environment, plays an essential role in many aspects of cognition, including speech segmentation and language acquisition. A key component of statistical learning in a linguistic context is the perceptual binding of adjacent individual units (e.g., syllables) into integrated composites (e.g., multisyllabic words). A second, conceptually dissociable component of statistical learning is the memory storage of these integrated representations. Here we examine whether these two dissociable components of statistical learning are differentially impacted by top-down, voluntary attentional resources. Learners' attention was either focused towards or diverted from a speech stream made up of repeating nonsense words. Building on our previous findings, we quantified the online perceptual binding of individual syllables into component words using an EEG-based neural entrainment measure. Following exposure, statistical learning was assessed using offline tests, sensitive to both perceptual binding and memory storage. Neural measures verified that our manipulation of selective attention successfully reduced limited-capacity resources to the speech stream. Diverting attention away from the speech stream did not alter neural entrainment to the component words or post-exposure familiarity ratings, but did impact performance on an indirect reaction-time based memory test. We conclude that theoretically dissociable components of statistically learning are differentially impacted by attention and top-down processing resources. A reduction in attention to the speech stream may impede memory storage of the component words. In contrast, the moment-by-moment perceptual binding of speech regularities can occur even while learners' attention is focused on a demanding concurrent task, and we found no evidence that selective attention modulates this process. These results suggest that learners can acquire basic statistical properties of language without directly focusing on the speech input, potentially opening up previously overlooked opportunities for language learning, particularly in adult learners.

Statistical Speech Segmentation in Tone Languages: The Role of Lexical Tones.

Research has demonstrated distinct roles for consonants and vowels in speech processing. For example, consonants have been shown to support lexical processes, such as the segmentation of speech based on transitional probabilities (TPs), more effectively than vowels. Theory and data so far, however, have considered only non-tone languages, that is to say, languages that lack contrastive lexical tones. In the present work, we provide a first investigation of the role of consonants and vowels in statistical speech segmentation by native speakers of Cantonese, as well as assessing how tones modulate the processing of vowels. Results show that Cantonese speakers are unable to use statistical cues carried by consonants for segmentation, but they can use cues carried by vowels. This difference becomes more evident when considering tone-bearing vowels. Additional data from speakers of Russian and Mandarin suggest that the ability of Cantonese speakers to segment streams with statistical cues carried by tone-bearing vowels extends to other tone languages, but is much reduced in speakers of non-tone languages.

Neurophysiological evidence for the interplay of speech segmentation and word-referent mapping during novel word learning.

Learning a new language requires the identification of word units from continuous speech (the speech segmentation problem) and mapping them onto conceptual representation (the word to world mapping problem). Recent behavioral studies have revealed that the statistical properties found within and across modalities can serve as cues for both processes. However, segmentation and mapping have been largely studied separately, and thus it remains unclear whether both processes can be accomplished at the same time and if they share common neurophysiological features. To address this question, we recorded EEG of 20 adult participants during both an audio alone speech segmentation task and an audiovisual word-to-picture association task. The participants were tested for both the implicit detection of online mismatches (structural auditory and visual semantic violations) as well as for the explicit recognition of words and word-to-picture associations. The ERP results from the learning phase revealed a delayed learning-related fronto-central negativity (FN400) in the audiovisual condition compared to the audio alone condition. Interestingly, while online structural auditory violations elicited clear MMN/N200 components in the audio alone condition, visual-semantic violations induced meaning-related N400 modulations in the audiovisual condition. The present results support the idea that speech segmentation and meaning mapping can take place in parallel and act in synergy to enhance novel word learning.

High-frequency neural activity predicts word parsing in ambiguous speech streams.

During speech listening, the brain parses a continuous acoustic stream of information into computational units (e.g., syllables or words) necessary for speech comprehension. Recent neuroscientific hypotheses have proposed that neural oscillations contribute to speech parsing, but whether they do so on the basis of acoustic cues (bottom-up acoustic parsing) or as a function of available linguistic representations (top-down linguistic parsing) is unknown. In this magnetoencephalography study, we contrasted acoustic and linguistic parsing using bistable speech sequences. While listening to the speech sequences, participants were asked to maintain one of the two possible speech percepts through volitional control. We predicted that the tracking of speech dynamics by neural oscillations would not only follow the acoustic properties but also shift in time according to the participant's conscious speech percept. Our results show that the latency of high-frequency activity (specifically, beta and gamma bands) varied as a function of the perceptual report. In contrast, the phase of low-frequency oscillations was not strongly affected by top-down control. Whereas changes in low-frequency neural oscillations were compatible with the encoding of prelexical segmentation cues, high-frequency activity specifically informed on an individual's conscious speech percept.