The time-course of linguistic interference during native and non-native speech-in-speech listening.

Recognizing speech in a noisy background is harder when the background is time-forward than for time-reversed speech, a masker direction effect, and harder when the masker is in a known rather than an unknown language, indicating linguistic interference. We examined the masker direction effect when the masker was a known vs unknown language and calculated performance over 50 trials to assess differential masker adaptation. In experiment 1, native English listeners transcribing English sentences showed a larger masker direction effect with English than Mandarin maskers. In experiment 2, Mandarin non-native speakers of English transcribing Mandarin sentences showed a mirror pattern. Both experiments thus support the target-masker linguistic similarity hypothesis, where interference is maximal when target and masker languages are the same. In experiment 3, Mandarin non-native speakers of English transcribing English sentences showed comparable results for English and Mandarin maskers. Non-native listening is therefore consistent with the known-language interference hypothesis, where interference is maximal when the masker language is known to the listener, whether or not it matches the target language. A trial-by-trial analysis showed that the masker direction effect increased over time during native listening but not during non-native listening. The results indicate different target-to-masker streaming strategies during native and non-native speech-in-speech listening.

Phonetic categorization and discrimination of voice onset time under divided attention.

Event durations are perceived to be shorter under divided attention. "Time shrinkage" is thought to be due to rapid attentional switches between tasks, leading to a loss of input samples, and hence, an under-estimation of duration. However, few studies have considered whether this phenomenon applies to durations relevant to time-based phonetic categorization. In this study, participants categorized auditory stimuli varying in voice onset time (VOT) as /É¡/ or /k/. They did so under focused attention (auditory task alone) or while performing a low-level visual task at the same time (divided attention). Under divided attention, there was increased response imprecision but no bias toward hearing /É¡/, the shorter-VOT sound. It is concluded that sample loss under divided attention does not apply to the perception of phonetic contrasts within the VOT range.

Effects of Cognitive Load on Pure-Tone Audiometry Thresholds in Younger and Older Adults.

OBJECTIVES: Cognitive load (CL) impairs listeners' ability to comprehend sentences, recognize words, and identify speech sounds. Recent findings suggest that this effect originates in a disruption of low-level perception of acoustic details. Here, we attempted to quantify such a disruption by measuring the effect of CL (a two-back task) on pure-tone audiometry (PTA) thresholds. We also asked whether the effect of CL on PTA was greater in older adults, on account of their reduced ability to divide cognitive resources between simultaneous tasks. To specify the mechanisms and representations underlying the interface between auditory and cognitive processes, we contrasted CL requiring visual encoding with CL requiring auditory encoding. Finally, the link between the cost of performing PTA under CL, working memory, and speech-in-noise (SiN) perception was investigated and compared between younger and older participants. DESIGN: Younger and older adults (44 in each group) did a PTA test at 0.5, 1, 2, and 4 kHz pure tones under CL and no CL. CL consisted of a visual two-back task running throughout the PTA test. The two-back task involved either visual encoding of the stimuli (meaningless images) or subvocal auditory encoding (a rhyme task on written nonwords). Participants also underwent a battery of SiN tests and a working memory test (letter number sequencing). RESULTS: Younger adults showed elevated PTA thresholds under CL, but only when CL involved subvocal auditory encoding. CL had no effect when it involved purely visual encoding. In contrast, older adults showed elevated thresholds under both types of CL. When present, the PTA CL cost was broadly comparable in younger and older adults (approximately 2 dB HL). The magnitude of PTA CL cost did not correlate significantly with SiN perception or working memory in either age group. In contrast, PTA alone showed strong links to both SiN and letter number sequencing in older adults. CONCLUSIONS: The results show that CL can exert its effect at the level of hearing sensitivity. However, in younger adults, this effect is only found when CL involves auditory mental representations. When CL involves visual representations, it has virtually no impact on hearing thresholds. In older adults, interference is found in both conditions. The results suggest that hearing progresses from engaging primarily modality-specific cognition in early adulthood to engaging cognition in a more undifferentiated way in older age. Moreover, hearing thresholds measured under CL did not predict SiN perception more accurately than standard PTA thresholds.

Cognitive load elevates discrimination thresholds of duration, intensity, and f0 for a synthesized vowel.

Dual-tasking negatively impacts on speech perception by raising cognitive load (CL). Previous research has shown that CL increases reliance on lexical knowledge and decreases reliance on phonetic detail. Less is known about the effect of CL on the perception of acoustic dimensions below the phonetic level. This study tested the effect of CL on the ability to discriminate differences in duration, intensity, and fundamental frequency of a synthesized vowel. A psychophysical adaptive procedure was used to obtain just noticeable differences (JNDs) on each dimension under load and no load. Load was imposed by N-back tasks at two levels of difficulty (one-back, two-back) and under two types of load (images, nonwords). Compared to a control condition with no CL, all N-back conditions increased JNDs across the three dimensions. JNDs were also higher under two-back than one-back load. Nonword load was marginally more detrimental than image load for intensity and fundamental frequency discrimination. Overall, the decreased auditory acuity demonstrates that the effect of CL on the listening experience can be traced to distortions in the perception of core auditory dimensions.

Lexical knowledge boosts statistically-driven speech segmentation.

The hypothesis that known words can serve as anchors for discovering new words in connected speech has computational and empirical support. However, evidence for how the bootstrapping effect of known words interacts with other mechanisms of lexical acquisition, such as statistical learning, is incomplete. In 3 experiments, we investigated the consequences of introducing a known word in an artificial language with no segmentation cues other than cross-syllable transitional probabilities. We started with an artificial language containing 4 trisyllabic novel words and observed standard above-chance performance in a subsequent recognition memory task. We then replaced 1 of the 4 novel words with a real word (tomorrow) and noted improved segmentation of the other 3 novel words. This improvement was maintained when the real word was a different length to the novel words (philosophy), ruling out an explanation based on metrical expectation. The improvement was also maintained when the word was added to the 4 original novel words rather than replacing 1 of them. Together, these results show that known words in an otherwise meaningless stream serve as anchors for discovering new words. In interpreting the results, we contrast a mechanism where the lexical boost is merely the consequence of attending to the edges of known words, with a mechanism where known words enhance sensitivity to transitional probabilities more generally. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Statistical learning for speech segmentation: Age-related changes and underlying mechanisms.

Statistical learning (SL) is a powerful learning mechanism that supports word segmentation and language acquisition in infants and young adults. However, little is known about how this ability changes over the life span and interacts with age-related cognitive decline. The aims of this study were to: (a) examine the effect of aging on speech segmentation by SL, and (b) explore core mechanisms underlying SL. Across four testing sessions, young, middle-aged, and older adults were exposed to continuous speech streams at two different speech rates, both with and without cognitive load. Learning was assessed using a two-alterative forced-choice task in which words from the stream were pitted against either part-words, which occurred across word boundaries in the stream, or nonwords, which never appeared in the stream. Participants also completed a battery of cognitive tests assessing working memory and executive functions. The results showed that speech segmentation by SL was remarkably resilient to aging, although age effects were visible in the more challenging conditions, namely, when words had to be discriminated from part-words, which required the formation of detailed phonological representations, and when SL was performed under cognitive load. Moreover, an analysis of the cognitive test data indicated that performance against part-words was predicted mostly by memory updating, whereas performance against nonwords was predicted mostly by working memory storage capacity. Taken together, the data show that SL relies on a combination of implicit and explicit skills, and that age effects on SL are likely to be linked to an age-related selective decline in memory updating. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Anterior paracingulate and cingulate cortex mediates the effects of cognitive load on speech sound discrimination.

Perceiving speech while performing another task is a common challenge in everyday life. How the brain controls resource allocation during speech perception remains poorly understood. Using functional magnetic resonance imaging (fMRI), we investigated the effect of cognitive load on speech perception by examining brain responses of participants performing a phoneme discrimination task and a visual working memory task simultaneously. The visual task involved holding either a single meaningless image in working memory (low cognitive load) or four different images (high cognitive load). Performing the speech task under high load, compared to low load, resulted in decreased activity in pSTG/pMTG and increased activity in visual occipital cortex and two regions known to contribute to visual attention regulation-the superior parietal lobule (SPL) and the paracingulate and anterior cingulate gyrus (PaCG, ACG). Critically, activity in PaCG/ACG was correlated with performance in the visual task and with activity in pSTG/pMTG: Increased activity in PaCG/ACG was observed for individuals with poorer visual performance and with decreased activity in pSTG/pMTG. Moreover, activity in a pSTG/pMTG seed region showed psychophysiological interactions with areas of the PaCG/ACG, with stronger interaction in the high-load than the low-load condition. These findings show that the acoustic analysis of speech is affected by the demands of a concurrent visual task and that the PaCG/ACG plays a role in allocating cognitive resources to concurrent auditory and visual information.

Is the superior verbal memory span of Mandarin speakers due to faster rehearsal?

It is well established that digit span in native Chinese speakers is atypically high. This is commonly attributed to a capacity for more rapid subvocal rehearsal for that group. We explored this hypothesis by testing a group of English-speaking native Mandarin speakers on digit span and word span in both Mandarin and English, together with a measure of speed of articulation for each. When compared to the performance of native English speakers, the Mandarin group proved to be superior on both digit and word spans while predictably having lower spans in English. This suggests that the Mandarin advantage is not limited to digits. Speed of rehearsal correlated with span performance across materials. However, this correlation was more pronounced for English speakers than for any of the Chinese measures. Further analysis suggested that speed of rehearsal did not provide an adequate account of differences between Mandarin and English spans or for the advantage of digits over words. Possible alternative explanations are discussed.

Sound specificity effects in spoken word recognition: The effect of integrality between words and sounds.

Recent evidence has shown that nonlinguistic sounds co-occurring with spoken words may be retained in memory and affect later retrieval of the words. This sound-specificity effect shares many characteristics with the classic voice-specificity effect. In this study, we argue that the sound-specificity effect is conditional upon the context in which the word and sound coexist. Specifically, we argue that, besides co-occurrence, integrality between words and sounds is a crucial factor in the emergence of the effect. In two recognition-memory experiments, we compared the emergence of voice and sound specificity effects. In Experiment 1 , we examined two conditions where integrality is high. Namely, the classic voice-specificity effect (Exp. 1a) was compared with a condition in which the intensity envelope of a background sound was modulated along the intensity envelope of the accompanying spoken word (Exp. 1b). Results revealed a robust voice-specificity effect and, critically, a comparable sound-specificity effect: A change in the paired sound from exposure to test led to a decrease in word-recognition performance. In the second experiment, we sought to disentangle the contribution of integrality from a mere co-occurrence context effect by removing the intensity modulation. The absence of integrality led to the disappearance of the sound-specificity effect. Taken together, the results suggest that the assimilation of background sounds into memory cannot be reduced to a simple context effect. Rather, it is conditioned by the extent to which words and sounds are perceived as integral as opposed to distinct auditory objects.

Separability of Tones and Rhymes in Chinese Speech Perception: Evidence from Perceptual Migrations.

This study used the perceptual-migration paradigm to explore whether Mandarin tones and syllable rhymes are processed separately during Mandarin speech perception. Following the logic of illusory conjunctions, we calculated the cross-ear migration of tones, rhymes, and their combination in Chinese and English listeners. For Chinese listeners, tones migrated more than rhymes. For English listeners, the opposite pattern was found. The results lend empirical support to autosegmental theory, which claims separability and mobility between tonal and segmental representations. They also provide evidence that such representations and their involvement in perception are deeply shaped by a listener's linguistic experience.

Magnified Neural Envelope Coding Predicts Deficits in Speech Perception in Noise.

Verbal communication in noisy backgrounds is challenging. Understanding speech in background noise that fluctuates in intensity over time is particularly difficult for hearing-impaired listeners with a sensorineural hearing loss (SNHL). The reduction in fast-acting cochlear compression associated with SNHL exaggerates the perceived fluctuations in intensity in amplitude-modulated sounds. SNHL-induced changes in the coding of amplitude-modulated sounds may have a detrimental effect on the ability of SNHL listeners to understand speech in the presence of modulated background noise. To date, direct evidence for a link between magnified envelope coding and deficits in speech identification in modulated noise has been absent. Here, magnetoencephalography was used to quantify the effects of SNHL on phase locking to the temporal envelope of modulated noise (envelope coding) in human auditory cortex. Our results show that SNHL enhances the amplitude of envelope coding in posteromedial auditory cortex, whereas it enhances the fidelity of envelope coding in posteromedial and posterolateral auditory cortex. This dissociation was more evident in the right hemisphere, demonstrating functional lateralization in enhanced envelope coding in SNHL listeners. However, enhanced envelope coding was not perceptually beneficial. Our results also show that both hearing thresholds and, to a lesser extent, magnified cortical envelope coding in left posteromedial auditory cortex predict speech identification in modulated background noise. We propose a framework in which magnified envelope coding in posteromedial auditory cortex disrupts the segregation of speech from background noise, leading to deficits in speech perception in modulated background noise.SIGNIFICANCE STATEMENT People with hearing loss struggle to follow conversations in noisy environments. Background noise that fluctuates in intensity over time poses a particular challenge. Using magnetoencephalography, we demonstrate anatomically distinct cortical representations of modulated noise in normal-hearing and hearing-impaired listeners. This work provides the first link among hearing thresholds, the amplitude of cortical representations of modulated sounds, and the ability to understand speech in modulated background noise. In light of previous work, we propose that magnified cortical representations of modulated sounds disrupt the separation of speech from modulated background noise in auditory cortex.

Auditory Verbal Working Memory as a Predictor of Speech Perception in Modulated Maskers in Listeners With Normal Hearing.

Purpose: Background noise can interfere with our ability to understand speech. Working memory capacity (WMC) has been shown to contribute to the perception of speech in modulated noise maskers. WMC has been assessed with a variety of auditory and visual tests, often pertaining to different components of working memory. This study assessed the relationship between speech perception in modulated maskers and components of auditory verbal working memory (AVWM) over a range of signal-to-noise ratios. Method: Speech perception in noise and AVWM were measured in 30 listeners (age range 31-67 years) with normal hearing. AVWM was estimated using forward digit recall, backward digit recall, and nonword repetition. Results: After controlling for the effects of age and average pure-tone hearing threshold, speech perception in modulated maskers was related to individual differences in the phonological component of working memory (as assessed by nonword repetition) but only in the least favorable signal-to-noise ratio. The executive component of working memory (as assessed by backward digit) was not predictive of speech perception in any conditions. Conclusions: AVWM is predictive of the ability to benefit from temporal dips in modulated maskers: Listeners with greater phonological WMC are better able to correctly identify sentences in modulated noise backgrounds.

How does cognitive load influence speech perception? An encoding hypothesis.

Two experiments investigated the conditions under which cognitive load exerts an effect on the acuity of speech perception. These experiments extend earlier research by using a different speech perception task (four-interval oddity task) and by implementing cognitive load through a task often thought to be modular, namely, face processing. In the cognitive-load conditions, participants were required to remember two faces presented before the speech stimuli. In Experiment 1, performance in the speech-perception task under cognitive load was not impaired in comparison to a no-load baseline condition. In Experiment 2, we modified the load condition minimally such that it required encoding of the two faces simultaneously with the speech stimuli. As a reference condition, we also used a visual search task that in earlier experiments had led to poorer speech perception. Both concurrent tasks led to decrements in the speech task. The results suggest that speech perception is affected even by loads thought to be processed modularly, and that, critically, encoding in working memory might be the locus of interference.

Speech segmentation by statistical learning is supported by domain-general processes within working memory.

The purpose of this study was to examine the extent to which working memory resources are recruited during statistical learning (SL). Participants were asked to identify novel words in an artificial speech stream where the transitional probabilities between syllables provided the only segmentation cue. Experiments 1 and 2 demonstrated that segmentation performance improved when the speech rate was slowed down, suggesting that SL is supported by some form of active processing or maintenance mechanism that operates more effectively under slower presentation rates. In Experiment 3 we investigated the nature of this mechanism by asking participants to perform a two-back task while listening to the speech stream. Half of the participants performed a two-back rhyme task designed to engage phonological processing, whereas the other half performed a comparable two-back task on un-nameable visual shapes. It was hypothesized that if SL is dependent only upon domain-specific processes (i.e., phonological rehearsal), the rhyme task should impair speech segmentation performance more than the shape task. However, the two loads were equally disruptive to learning, as they both eradicated the benefit provided by the slow rate. These results suggest that SL is supported by working-memory processes that rely on domain-general resources.

Beating the bounds: Localized timing cues to word segmentation.

Prosody facilitates perceptual segmentation of the speech stream into a sequence of words and phrases. With regard to speech timing, vowel lengthening is well established as a cue to an upcoming boundary, but listeners' exploitation of consonant lengthening for segmentation has not been systematically tested in the absence of other boundary cues. In a series of artificial language learning experiments, the impact of durational variation in consonants and vowels on listeners' extraction of novel trisyllables was examined. Language streams with systematic lengthening of word-initial consonants were better recalled than both control streams without localized lengthening and streams where word-initial syllable lengthening was confined to the vocalic rhyme. Furthermore, where vowel-consonant sequences were lengthened word-medially, listeners failed to learn the languages effectively. Thus the structural interpretation of lengthening effects depends upon their localization, in this case, a distinction between lengthening of the onset consonant and the vocalic syllable rhyme. This functional division is considered in terms of speech-rate-sensitive predictive mechanisms and listeners' expectations regarding the occurrence of syllable perceptual centres.

Divided attention disrupts perceptual encoding during speech recognition.

Performing a secondary task while listening to speech has a detrimental effect on speech processing, but the locus of the disruption within the speech system is poorly understood. Recent research has shown that cognitive load imposed by a concurrent visual task increases dependency on lexical knowledge during speech processing, but it does not affect lexical activation per se. This suggests that "lexical drift" under cognitive load occurs either as a post-lexical bias at the decisional level or as a secondary consequence of reduced perceptual sensitivity. This study aimed to adjudicate between these alternatives using a forced-choice task that required listeners to identify noise-degraded spoken words with or without the addition of a concurrent visual task. Adding cognitive load increased the likelihood that listeners would select a word acoustically similar to the target even though its frequency was lower than that of the target. Thus, there was no evidence that cognitive load led to a high-frequency response bias. Rather, cognitive load seems to disrupt sublexical encoding, possibly by impairing perceptual acuity at the auditory periphery.

Phoneme categorization and discrimination in younger and older adults: a comparative analysis of perceptual, lexical, and attentional factors.

This study investigates the extent to which age-related language processing difficulties are due to a decline in sensory processes or to a deterioration of cognitive factors, specifically, attentional control. Two facets of attentional control were examined: inhibition of irrelevant information and divided attention. Younger and older adults were asked to categorize the initial phoneme of spoken syllables ("Was it m or n?"), trying to ignore the lexical status of the syllables. The phonemes were manipulated to range in eight steps from m to n. Participants also did a discrimination task on syllable pairs ("Were the initial sounds the same or different?"). Categorization and discrimination were performed under either divided attention (concurrent visual-search task) or focused attention (no visual task). The results showed that even when the younger and older adults were matched on their discrimination scores: (1) the older adults had more difficulty inhibiting lexical knowledge than did younger adults, (2) divided attention weakened lexical inhibition in both younger and older adults, and (3) divided attention impaired sound discrimination more in older than younger listeners. The results confirm the independent and combined contribution of sensory decline and deficit in attentional control to language processing difficulties associated with aging. The relative weight of these variables and their mechanisms of action are discussed in the context of theories of aging and language.

Extrinsic cognitive load impairs low-level speech perception.

Recent research has suggested that the extrinsic cognitive load generated by performing a nonlinguistic visual task while perceiving speech increases listeners' reliance on lexical knowledge and decreases their capacity to perceive phonetic detail. In the present study, we asked whether this effect is accounted for better at a lexical or a sublexical level. The former would imply that cognitive load directly affects lexical activation but not perceptual sensitivity; the latter would imply that increased lexical reliance under cognitive load is only a secondary consequence of imprecise or incomplete phonetic encoding. Using the phoneme restoration paradigm, we showed that perceptual sensitivity decreases (i.e., phoneme restoration increases) almost linearly with the effort involved in the concurrent visual task. However, cognitive load had only a minimal effect on the contribution of lexical information to phoneme restoration. We concluded that the locus of extrinsic cognitive load on the speech system is perceptual rather than lexical. Mechanisms by which cognitive load increases tolerance to acoustic imprecision and broadens phonemic categories were discussed.

Segmentation cues in conversational speech: robust semantics and fragile phonotactics.

Multiple cues influence listeners' segmentation of connected speech into words, but most previous studies have used stimuli elicited in careful readings rather than natural conversation. Discerning word boundaries in conversational speech may differ from the laboratory setting. In particular, a speaker's articulatory effort - hyperarticulation vs. hypoarticulation (H&H) - may vary according to communicative demands, suggesting a compensatory relationship whereby acoustic-phonetic cues are attenuated when other information sources strongly guide segmentation. We examined how listeners' interpretation of segmentation cues is affected by speech style (spontaneous conversation vs. read), using cross-modal identity priming. To elicit spontaneous stimuli, we used a map task in which speakers discussed routes around stylized landmarks. These landmarks were two-word phrases in which the strength of potential segmentation cues - semantic likelihood and cross-boundary diphone phonotactics - was systematically varied. Landmark-carrying utterances were transcribed and later re-recorded as read speech. Independent of speech style, we found an interaction between cue valence (favorable/unfavorable) and cue type (phonotactics/semantics). Thus, there was an effect of semantic plausibility, but no effect of cross-boundary phonotactics, indicating that the importance of phonotactic segmentation may have been overstated in studies where lexical information was artificially suppressed. These patterns were unaffected by whether the stimuli were elicited in a spontaneous or read context, even though the difference in speech styles was evident in a main effect. Durational analyses suggested speaker-driven cue trade-offs congruent with an H&H account, but these modulations did not impact on listener behavior. We conclude that previous research exploiting read speech is reliable in indicating the primacy of lexically based cues in the segmentation of natural conversational speech.

How stable are acoustic metrics of contrastive speech rhythm?

Acoustic metrics of contrastive speech rhythm, based on vocalic and intervocalic interval durations, are intended to capture stable typological differences between languages. They should consequently be robust to variation between speakers, sentence materials, and measurers. This paper assesses the impact of these sources of variation on the metrics %V (proportion of utterance comprised of vocalic intervals), VarcoV (rate-normalized standard deviation of vocalic interval duration), and nPVI-V (a measure of the durational variability between successive pairs of vocalic intervals). Five measurers analyzed the same corpus of speech: five sentences read by six speakers of Standard Southern British English. Differences between sentences were responsible for the greatest variation in rhythm scores. Inter-speaker differences were also a source of significant variability. However, there was relatively little variation due to segmentation differences between measurers following an agreed protocol. An automated phone alignment process was also used: Rhythm scores thus derived showed good agreement with the human measurers. A number of recommendations for researchers wishing to exploit contrastive rhythm metrics are offered in conclusion.