Evaluating Speaker-Listener Cognitive Effort in Speech Communication Through Brain-to-Brain Synchrony: A Pilot Functional Near-Infrared Spectroscopy Investigation.

PURPOSE: We explore a new approach to the study of cognitive effort involved in listening to speech by measuring the brain activity in a listener in relation to the brain activity in a speaker. We hypothesize that the strength of this brain-to-brain synchrony (coupling) reflects the magnitude of cognitive effort involved in verbal communication and includes both listening effort and speaking effort. We investigate whether interbrain synchrony is greater in native-to-native versus native-to-nonnative communication using functional near-infrared spectroscopy (fNIRS). METHOD: Two speakers participated, a native speaker of American English and a native speaker of Korean who spoke English as a second language. Each speaker was fitted with the fNIRS cap and told short stories. The native English speaker provided the English narratives, and the Korean speaker provided both the nonnative (accented) English and Korean narratives. In separate sessions, fNIRS data were obtained from seven English monolingual participants ages 20-24 years who listened to each speaker's stories. After listening to each story in native and nonnative English, they retold the content, and their transcripts and audio recordings were analyzed for comprehension and discourse fluency, measured in the number of hesitations and articulation rate. No story retellings were obtained for narratives in Korean (an incomprehensible language for English listeners). Utilizing fNIRS technique termed sequential scanning, we quantified the brain-to-brain synchronization in each speaker-listener dyad. RESULTS: For native-to-native dyads, multiple brain regions associated with various linguistic and executive functions were activated. There was a weaker coupling for native-to-nonnative dyads, and only the brain regions associated with higher order cognitive processes and functions were synchronized. All listeners understood the content of all stories, but they hesitated significantly more when retelling stories told in accented English. The nonnative speaker hesitated significantly more often than the native speaker and had a significantly slower articulation rate. There was no brain-to-brain coupling during listening to Korean, indicating a break in communication when listeners failed to comprehend the speaker. CONCLUSIONS: We found that effortful speech processing decreased interbrain synchrony and delayed comprehension processes. The obtained brain-based and behavioral patterns are consistent with our proposal that cognitive effort in verbal communication pertains to both the listener and the speaker and that brain-to-brain synchrony can be an indicator of differences in their cumulative communicative effort. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.25452142.

Introduction to the special issue on perception and production of sounds in the high-frequency range of human speecha).

The frequency range audible to humans can extend from 20 Hz to 20 kHz, but only a portion of this range-the lower end up to 8 kHz-has been systematically explored because extended high-frequency (EHF) information above this low range has been considered unnecessary for speech comprehension. This special issue presents a collection of research studies exploring the presence of EHF information in the acoustic signal and its perceptual utility. The papers address the role of EHF hearing in auditory perception, the impact of EHF hearing loss on speech perception in specific populations and occupational settings, the importance of EHF in speech recognition and in providing speaker-related information, the utility of acoustic EHF energy in fricative sounds, and ultrasonic vocalizations in mice in relation to human hearing. Collectively, the research findings offer new insights and converge in showing that not only is EHF energy present in the speech spectrum, but listeners can utilize EHF cues in speech processing and recognition, and EHF hearing loss has detrimental effects on perception of speech and non-speech sounds. Together, this collection challenges the conventional notion that EHF information has minimal functional significance.

Dialect and gender perception in relation to the intelligibility of low-pass and high-pass filtered spontaneous speecha).

Most cues to speech intelligibility are within a narrow frequency range, with its upper limit not exceeding 4 kHz. It is still unclear whether speaker-related (indexical) information is available past this limit or how speaker characteristics are distributed at frequencies within and outside the intelligibility range. Using low-pass and high-pass filtering, we examined the perceptual salience of dialect and gender cues in both intelligible and unintelligible speech. Setting the upper frequency limit at 11 kHz, spontaneously produced unique utterances (n = 400) from 40 speakers were high-pass filtered with frequency cutoffs from 0.7 to 5.56 kHz and presented to listeners for dialect and gender identification and intelligibility evaluation. The same material and experimental procedures were used to probe perception of low-pass filtered and unmodified speech with cutoffs from 0.5 to 1.1 kHz. Applying statistical signal detection theory analyses, we found that cues to gender were well preserved at low and high frequencies and did not depend on intelligibility, and the redundancy of gender cues at higher frequencies reduced response bias. Cues to dialect were relatively strong at low and high frequencies; however, most were in intelligible speech, modulated by a differential intelligibility advantage of male and female speakers at low and high frequencies.

Perception of indexical cues in speech by children and adults with and without dyslexia: Regional dialect and gender identification.

Auditory research in developmental dyslexia proposes that deficient auditory processing of speech underlies difficulties with reading and spelling. Focusing predominantly on phonological processing, studies have not yet addressed the role of the speaker-related (indexical) properties of speech that enable the formation of phonological representations. Here, we assess auditory processing of indexical characteristics cueing a speaker's regional dialect and gender to determine whether dyslexia constraints recognition of dialect features and voice gender. Adults and children aged 11-14 years with dyslexia and their age-matched controls responded to 360 unique sentences extracted from spontaneous conversations of 40 speakers. In addition to the original unprocessed speech, there were two focused filtered conditions (using lowpass filtering at 400 Hz and 8-channel noise vocoding) probing listeners' responses to segmental and prosodic cues. Compared with controls, both groups with dyslexia were significantly limited in their abilities to recognize dialect features from either set of cues. The results for gender suggest that their comparatively worse gender recognition in the noise-vocoded condition was possibly related to poor temporal resolution. We propose that the deficient processing of indexical cues by individuals with dyslexia originates in peripheral auditory processes, of which impaired processing of relevant temporal cues in amplitude envelope is a likely candidate.

Variability in within-category implementation of stop consonant voicing in American English-speaking children.

The development of stop consonant voicing in English-speaking children has been documented as a progressive mastery of phonological contrast, but implementation of voicing within one voicing category has not been systematically examined. This study provides a comprehensive account of structured variability in phonetic realization of /b/ in running speech by 8-12-year-old American children (n = 48) when compared to adults (n = 36). The stop always occurred word-initially, was followed by either a voiced or voiceless coda, and its position varied in a sentence, which created systematic conditions to examine acoustic variability in closure duration (CD) and voicing during the closure (VDC) stemming from phonetic context and prosodic prominence. Children demonstrated command of long-distance anticipatory coarticulation, providing evidence that information about coda voicing is distributed over an entire monosyllabic word and is available in the onset stop. They also manifested covariation of cues to stop voicing and command of prosodic variation, despite greater random variability, greater CD, reduced VDC, and exaggerated execution of sentential focus when compared to adults. Controlling for regional variation, dialect was a significant predictor for adults but not for children, who no longer adhered to the marked local variants in their implementation of stop voicing.

Perception of local and non-local vowels by adults and children in the South.

This study assessed the ability of Southern listeners to accommodate extensive talker variability in identifying vowels in their local Appalachian community in the context of sound change. Building on prior work, the current experiment targeted a subset of spectrally overlapping vowels in local and two non-local varieties to establish whether adult and child listeners will demonstrate the local dialect advantage. Listeners responded to isolated target words, which minimized the interaction of multiple linguistic and dialect-specific features. For most vowel categories, the local dialect advantage was not demonstrated. However, adult listeners showed sensitivity to generational changes, indicating their familiarity with the local norms. A differential response pattern in children suggests that children perceived the vowels through the lens of their own experience with vowel production, representing a sound change in the community. Compared with the adults, children also relied more on stress cues, with increased confusions when the vowels were unstressed. The study provides evidence that identification accuracy is dependent upon the robustness of cues in individual vowel categories-whether local or non-local-and suggests that the bottom-up processes underlying phonetic vowel categorization in isolated monosyllables can interact with the top-down processing of dialect- and talker-specific information.

The old, the new, and the in-between: Preadolescents' use of stylistic variation in speech in projecting their own identity in a culturally changing environment.

Cultural learning begins early, with infants' and young children's initial imitations of group-specific local behaviors. Comparatively little is known about cultural development in older children, whose more advanced socio-cognitive skills can moderate their decisions about adherence to the established cultural conventions and acceptance of new norms. Focusing on the acquisition of a regional dialect, the current study was conducted in a small community in western North Carolina, whose rich Appalachian heritage grew from distinctive cultural and living traditions. The region has gradually opened up to outside influences and the local culture is now shifting toward mainstream American socio-cultural norms. The study sought to determine how preadolescents positioned themselves in this socio-culturally changing environment. Using detailed acoustic analysis to measure stylistic variation in speech in 9-12-year-olds and perceptual ratings to verify its salience, we examined the pronunciation of the vowel /ai/ to test children's adherence to the old Appalachian identity marker (the monophthong) and their acceptance of the modern American society (the diphthong). As an innovation, children created an intermediate phonetic variant that reduced the pronunciation differences between the old and modern patterns. Demonstrating the ability to adapt speech style to context, they increased the degree of diphthongization in this /ai/-variant in careful speech (reading), and reduced it in casual conversations. Girls' productions were more diphthongal than were boys' in reading but not in conversations. The new variant in children represents regional dialect levelling, and likely results from their accommodation to the changing environment, which promotes reduction of old marked forms.

Regional Variation in Fundamental Frequency of American English Vowels.

We examined whether the fundamental frequency (f0) of vowels is influenced by regional variation, aiming to (1) establish how the relationship between vowel height and f0 ("intrinsic f0") is utilized in regional vowel systems and (2) determine whether regional varieties differ in their implementation of the effects of phonetic context on f0 variations. An extended set of acoustic measures explored f0 in vowels in isolated tokens (experiment 1) and in connected speech (experiment 2) from 36 women representing 3 different varieties of American English. Regional differences were found in f0 shape in isolated tokens, in the magnitude of intrinsic f0 difference between high and low vowels, in the nature of f0 contours in stressed vowels, and in the completion of f0 contours in the context of coda voicing. Regional varieties utilize f0 control in vowels in different ways, including regional f0 ranges and variation in f0 shape.

Reconceptualizing the vowel space in analyzing regional dialect variation and sound change in American English.

Vowel space area (VSA) calculated on the basis of corner vowels has emerged as a metric for the study of regional variation, speech intelligibility and speech development. This paper gives an evaluation of the basic assumptions underlying both the concept of the vowel space and the utility of the VSA in making cross-dialectal and sound change comparisons. Using cross-generational data from 135 female speakers representing three distinct dialects of American English, the first step was to establish that the vowel quadrilateral fails as a metric in the context of dialect variation. The next step was to examine the efficacy of more complete assessments of VSA represented by the convex hull and the concave hull. Despite the improvement over the quadrilateral, both metrics yielded inconsistent estimates of VSA. This paper then explores the possibility that regional variation can be characterized more effectively if formant dynamics and the resulting spectral overlap were also considered in defining the space. The proposed formant density approach showed that the working space may be common to all dialects but the differences are in the internal distribution of spectral density regions that define dialect-specific "usage" of the acoustic space. The dialect-inherent distribution of high and low density regions is largely shaped by sound change.

Dyslexia Limits the Ability to Categorize Talker Dialect.

Purpose: The purpose of this study was to determine whether the underlying phonological impairment in dyslexia is associated with a deficit in categorizing regional dialects. Method: Twenty adults with dyslexia, 20 school-age children with dyslexia, and 40 corresponding control listeners with average reading ability listened to sentences produced by multiple talkers (both sexes) representing two dialects: Midland dialect in Ohio (same as listeners' dialect) and Southern dialect in Western North Carolina. Participants' responses were analyzed using signal detection theory. Results: Listeners with dyslexia were less sensitive to talker dialect than listeners with average reading ability. Children were less sensitive to dialect than adults. Under stimulus uncertainty, listeners with average reading ability were biased toward Ohio dialect, whereas listeners with dyslexia were unbiased in their responses. Talker sex interacted with sensitivity and bias differently for listeners with dyslexia than for listeners with average reading ability. The correlations between dialect sensitivity and phonological memory scores were strongest for adults with dyslexia. Conclusions: The results imply that the phonological deficit in dyslexia arises from impaired access to intact phonological representations rather than from poorly specified representations. It can be presumed that the impeded access to implicit long-term memory representations for indexical (dialect) information is due to less efficient operations in working memory, including deficiencies in utilizing talker normalization processes.

Intrinsic fundamental frequency of vowels is moderated by regional dialect.

There has been a long-standing debate whether the intrinsic fundamental frequency (IF0) of vowels is an automatic consequence of articulation or whether it is independently controlled by speakers to perceptually enhance vowel contrasts along the height dimension. This paper provides evidence from regional variation in American English that IF0 difference between high and low vowels is, in part, controlled and varies across dialects. The sources of this F0 control are socio-cultural and cannot be attributed to differences in the vowel inventory size. The socially motivated enhancement was found only in prosodically prominent contexts.

Variation in Vowel Duration Among Southern African American English Speakers.

PURPOSE: Atypical duration of speech segments can signal a speech disorder. In this study, we examined variation in vowel duration in African American English (AAE) relative to White American English (WAE) speakers living in the same dialect region in the South to characterize the nature of systematic variation between the 2 groups. The goal was to establish whether segmental durations in minority populations differ from the well-established patterns in mainstream populations. METHOD: Participants were 32 AAE and 32 WAE speakers differing in age who, in their childhood, attended either segregated (older speakers) or integrated (younger speakers) public schools. Speech materials consisted of 14 vowels produced in hVd-frame. RESULTS: AAE vowels were significantly longer than WAE vowels. Vowel duration did not differ as a function of age. The temporal tense-lax contrast was minimized for AAE relative to WAE. Vowels produced by females were significantly longer than vowels produced by males for both AAE and WAE. CONCLUSIONS: African American speakers should be expected to produce longer vowels relative to White speakers in a common geographic area. These longer durations are not deviant but represent a typical feature of AAE. This finding has clinical importance in guiding assessments of speech disorders in AAE speakers.

Vowel development in an emergent Mandarin-English bilingual child: a longitudinal study.

This longitudinal case study documents the emergence of bilingualism in a young monolingual Mandarin boy on the basis of an acoustic analysis of his vowel productions recorded via a picture-naming task over 20 months following his enrollment in an all-English (L2) preschool at the age of 3;7. The study examined (1) his initial L2 vowel space, (2) the process of L1-L2 separation, and (3) his L1 vowel system in relation to L2. The child initially utilized his L1 base in building the L2 vowel system. The L1-L2 separation started from a drastic restructuring of his working vowel space to create maximal contrast between the two languages. Meanwhile, L1 developmental processes and influence of L2 on L1 were also in effect. The developmental profile of this child uncovered strategies sequential bilingual children may use to restructure their phonetic space and construct a new system of contrasts in L2.

The effects of indexical and phonetic variation on vowel perception in typically developing 9- to 12-year-old children.

PURPOSE The purpose of this study was to investigate how linguistic knowledge interacts with indexical knowledge in older children's perception under demanding listening conditions created by extensive talker variability. METHOD Twenty-five 9- to 12-year-old children, 12 from North Carolina (NC) and 13 from Wisconsin (WI), identified 12 vowels in isolated /hVd/ words produced by 120 talkers representing the 2 dialects (NC and WI), both genders, and 3 age groups (generations) of residents from the same geographic locations as the listeners. RESULTS Identification rates were higher for responses to talkers from the same dialect as the listeners and for female speech. Listeners were sensitive to systematic positional variations in vowels and their dynamic structure (formant movement) associated with generational differences in vowel pronunciation resulting from sound change in a speech community. Overall identification rate was 71.7%, which is 8.5% lower than for the adults responding to the same stimuli in Jacewicz and Fox (2012). CONCLUSION Typically developing older children were successful in dealing with both phonetic and indexical variation related to talker dialect, gender, and generation. They were less consistent than the adults, most likely because of less efficient encoding of acoustic-phonetic information in the speech of multiple talkers and relative inexperience with indexical variation.

The effects of concurrent cognitive load on phonological processing in adults who stutter.

PURPOSE: To determine whether phonological processing in adults who stutter (AWS) is disrupted by increased amounts of cognitive load in a concurrent attention-demanding task. METHOD: Nine AWS and 9 adults who do not stutter (AWNS) participated. Using a dual-task paradigm, the authors presented word pairs for rhyme judgments and, concurrently, letter strings for memory recall. The rhyme judgment task manipulated rhyming type (rhyming/nonrhyming) and orthographic representation (similar/dissimilar). The memory recall task varied stimulus complexity (no letters, 3 letters, 5 letters). Rhyme judgment accuracy and reaction time (RT) were used to assess phonological processing, and letter recall accuracy was used to measure memory recall. RESULTS: For rhyme judgments, AWS were as accurate as AWNS, and the increase in the cognitive load did not affect rhyme judgment accuracy of either group. Significant group differences were found in RTs (delays by AWS were 241 ms greater). RTs of AWS were also slower in the most demanding rhyme condition and varied with the complexity of the memory task. Accuracy of letter recall of AWS was comparatively worse in the most demanding 5-letter condition. CONCLUSION: Phonological and cognitive processing of AWS is more vulnerable to disruptions caused by increased amounts of cognitive load in concurrent attention-demanding tasks.

The effects of cross-generational and cross-dialectal variation on vowel identification and classification.

Cross-generational and cross-dialectal variation in vowels among speakers of American English was examined in terms of vowel identification by listeners and vowel classification using pattern recognition. Listeners from Western North Carolina and Southeastern Wisconsin identified 12 vowel categories produced by 120 speakers stratified by age (old adults, young adults, and children), gender, and dialect. The vowels /ɝ, o, ʊ, u/ were well identified by both groups of listeners. The majority of confusions were for the front /i, ɪ, e, ɛ, æ/, the low back /ɑ, ɔ/ and the monophthongal North Carolina /aɪ/. For selected vowels, generational differences in acoustic vowel characteristics were perceptually salient, suggesting listeners' responsiveness to sound change. Female exemplars and native-dialect variants produced higher identification rates. Linear discriminant analyses which examined dialect and generational classification accuracy showed that sampling the formant pattern at vowel midpoint only is insufficient to separate the vowels. Two sample points near onset and offset provided enough information for successful classification. The models trained on one dialect classified the vowels from the other dialect with much lower accuracy. The results strongly support the importance of dynamic information in accurate classification of cross-generational and cross-dialectal variations.

Vowel change across three age groups of speakers in three regional varieties of American English.

This acoustic study examines sound (vowel) change in apparent time across three successive generations of 123 adult female speakers ranging in age from 20 to 65 years old, representing three regional varieties of American English, typical of western North Carolina, central Ohio and southeastern Wisconsin. A set of acoustic measures characterized the dynamic nature of formant trajectories, the amount of spectral change over the course of vowel duration and the position of the spectral centroid. The study found a set of systematic changes to /I, ε, æ/ including positional changes in the acoustic space (mostly lowering of the vowels) and significant variation in formant dynamics (increased monophthongization). This common sound change is evident in both emphatic (articulated clearly) and nonemphatic (casual) productions and occurs regardless of dialect-specific vowel dispersions in the vowel space. The cross-generational and cross-dialectal patterns of variation found here support an earlier report by Jacewicz, Fox, and Salmons (2011) which found this recent development in these three dialect regions in isolated citation-form words. While confirming the new North American Shift in different styles of production, the study underscores the importance of addressing the stress-related variation in vowel production in a careful and valid assessment of sound change.

Auditory spectral integration in the perception of static vowels.

PURPOSE: To evaluate potential contributions of broadband spectral integration in the perception of static vowels. Specifically, can the auditory system infer formant frequency information from changes in the intensity weighting across harmonics when the formant itself is missing? Does this type of integration produce the same results in the lower (first formant [F1]) and higher (second formant [F2]) regions? Does the spacing between the spectral components affect a listener's ability to integrate the acoustic cues? METHOD: Twenty young listeners with normal hearing identified synthesized vowel-like stimuli created for adjustments in the F1 region (/Λ/-/α/, /i/-/ε/) and in the F2 region (/Λ/-/æ/). There were 2 types of stimuli: (a) 2-formant tokens and (b) tokens in which 1 formant was removed and 2 pairs of sine waves were inserted below and above the missing formant; the intensities of these harmonics were modified to cause variations in their spectral center of gravity (COG). The COG effects were tested over a wide range of frequencies. RESULTS: Obtained patterns were consistent with calculated changes to the spectral COG, in both the F1 and F2 regions. The spacing of the sine waves did not affect listeners' responses. CONCLUSION: The auditory system may perform broadband integration as a type of auditory wideband spectral analysis.

Regional dialect variation in the vowel systems of typically developing children.

PURPOSE: To investigate regional dialect variation in the vowel systems of typically developing 8- to 12-year-old children. METHOD: Thirteen vowels in isolated h_d words were produced by 94 children and 93 adults (males and females). All participants spoke American English and were born and raised in 1 of 3 distinct dialect regions in the United States: western North Carolina (Southern dialect), central Ohio (Midland dialect), and southeastern Wisconsin (Northern Midwestern dialect). Acoustic analysis included formant frequencies (F1 and F2) measured at 5 equidistant time points in a vowel and formant movement (trajectory length). RESULTS: Children's productions showed many dialect-specific features comparable to those in adult speakers, both in terms of vowel dispersion patterns and formant movement. Different features were also found, including systemic vowel changes, significant monophthongization of selected vowels, and greater formant movement in diphthongs. CONCLUSIONS: The acoustic results provide evidence for regional distinctiveness in children's vowel systems. Children acquire not only the systemic relations among vowels but also their dialect-specific patterns of formant dynamics. Directing attention to the regional variation in the production of American English vowels, this work may prove helpful in better understanding and interpreting the development of vowel categories and vowel systems in children.

Cross-generational vowel change in American English.

This study examines cross-generational changes in the vowel systems in central Ohio, southeastern Wisconsin and western North Carolina. Speech samples from 239 speakers, males and females, were divided into three age groups: grandparents (66-91 years old), parents (35-51) and children (8-12). Acoustic analysis of vowel dynamics (i.e., formant movement) was undertaken to explore variation in the amount of spectral change for each vowel. A robust set of cross-generational changes in /ɪ, ε, æ, ɑ/ was found within each dialect-specific vowel system, involving both their positions and dynamics. With each successive generation, /ɪ, ε, æ/ become increasingly monophthongized and /ɑ/ is diphthongized in children. These changes correspond to a general anticlockwise parallel rotation of vowels (with some exceptions in /ɪ/ and /ε/). Given the widespread occurrence of these parallel chain-like changes, we term this development the "North American Shift" which conforms to the general principles of chain shifting formulated by Labov (1994) and others.