Self-generated strategies in the phonological similarity effect.

Strategy use is an important source of individual differences during immediate serial reconstruction. However, not all strategies are equally suited for all tasks. Therefore, assessing participants' dynamic strategy selection across contexts is an important next step for reliable interpretation of individual differences in short-term memory span - in both experimental and clinical settings. Strategy use during reconstruction of phonologically similar and phonologically distinct word sets was directly assessed using a self-report questionnaire. In two experiments, participants reported consistent use of phonological strategies across word sets; however, participants reported additionally using non-phonological strategies (i.e., mental imagery and sentence generation) when tasked with remembering phonologically similar words. In particular, strategy selection was most impacted when the phonologically similar word set was either the only word set or the first word set participants received. When the phonologically similar lists were presented after a classic list of phonologically distinct words, participants continued using the phonological strategies that had been effective for the distinct lists. Moreover, in both experiments, accuracy of phonologically similar lists was better predicted by use of non-phonological strategies than use of phonological strategies. Specifically, reported use of verbalization or rehearsal did not predict accuracy, but participants who reported regularly using mental imagery and/or sentence generation (typically in conjunction with rehearsal) displayed greater serial memory for similar words. These results do not undermine the general assumptions of the phonological similarity effect, but they do indicate that its interpretation is less straightforward than previously thought.

Effects of Production Training With Ultrasound Biofeedback on Production and Perception of Second-Language English Tense-Lax Vowel Contrasts.

PURPOSE: This study investigated the efficacy of ultrasound biofeedback compared to a traditional acoustic input-only method in the second-language (L2) production training of English tense-lax vowel contrasts (/i-ɪ/ and /e-ɛ/). The relationship between the degrees of production and perception learning was also examined to explore a potential L2 production-perception link. METHOD: Using a pretest/training/posttest paradigm, two groups of native Mandarin speakers received three sessions of training, one with ultrasound and one with audio only. Perception performance was measured by accuracy in AX discrimination. Production performance was measured in terms of accuracy in repetition production and Euclidean distance between contrasting vowels. RESULTS: Both groups showed significant improvements in posttest production and perception tasks. Additionally, the training benefits were retained 6 weeks after training. The two groups, however, did not differ significantly in training gains; nor was there a significant correlation between training-induced changes in perception and production performance. CONCLUSIONS: The results indicate that ultrasound feedback is similarly effective as the audio-only training in improving the L2 production of English tense-lax contrasts and the benefits transferred to perception. The lack of correlation between production and perception gains suggests that the two modalities need to be separately trained to achieve desired L2 learning outcomes.

Tutorial: Using Visual-Acoustic Biofeedback for Speech Sound Training.

PURPOSE: This tutorial summarizes current practices using visual-acoustic biofeedback (VAB) treatment to improve speech outcomes for individuals with speech sound difficulties. Clinical strategies will focus on residual distortions of /ɹ/. METHOD: Summary evidence related to the characteristics of VAB and the populations that may benefit from this treatment are reviewed. Guidelines are provided for clinicians on how to use VAB with clients to identify and modify their productions to match an acoustic representation. The clinical application of a linear predictive coding spectrum is emphasized. RESULTS: Successful use of VAB requires several key factors including clinician and client comprehension of the acoustic representation, appropriate acoustic target and template selection, as well as appropriate selection of articulatory strategies, practice schedules, and feedback models to scaffold acquisition of new speech sounds. CONCLUSION: Integrating a VAB component in clinical practice offers additional intervention options for individuals with speech sound difficulties and often facilitates improved speech sound acquisition and generalization outcomes. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.21817722.

Classification of accurate and misarticulated /ɑr/ for ultrasound biofeedback using tongue part displacement trajectories.

Ultrasound biofeedback therapy (UBT), which incorporates real-time imaging of tongue articulation, has demonstrated generally positive speech remediation outcomes for individuals with residual speech sound disorder (RSSD). However, UBT requires high attentional demands and may therefore benefit from a simplified display of articulation targets that are easily interpretable and can be compared to real-time articulation. Identifying such targets requires automatic quantification and analysis of movement features relevant to accurate speech production. Our image-analysis program TonguePART automatically quantifies tongue movement as tongue part displacement trajectories from midsagittal ultrasound videos of the tongue, with real-time capability. The present study uses such displacement trajectories to compare accurate and misarticulated American-English rhotic /ɑr/ productions from 40 children, with degree of accuracy determined by auditory perceptual ratings. To identify relevant features of accurate articulation, support vector machine (SVM) classifiers were trained and evaluated on several candidate data representations. Classification accuracy was up to 85%, indicating that quantification of tongue part displacement trajectories captured tongue articulation characteristics that distinguish accurate from misarticulated production of /ɑr/. Regression models for perceptual ratings were also compared. The simplest data representation that retained high predictive ability, demonstrated by high classification accuracy and strong correlation between observed and predicted ratings, was displacements at the midpoint of /r/ relative to /ɑ/ for the tongue dorsum and blade. This indicates that movements of the dorsum and blade are especially relevant to accurate production of /r/, suggesting that a predictive parameter and biofeedback target based on this data representation may be usable for simplified UBT.

Effect of the Temporal Window of Integration of Speech Sound on Mismatch Negativity.

Speech-sound stimuli have a complex structure, and it is unclear how the brain processes them. An event-related potential (ERP), known as mismatch negativity (MMN), is elicited when an individual's brain detects a rare sound. In this study, MMNs were measured in response to an omitted segment of a complex sound consisting of a Japanese vowel. The results indicated that the latency from onset in the right hemisphere was significantly shorter than that in the frontal midline and left hemispheres during left ear stimulation. Additionally, the results of latency from omission showed that the latency of stimuli omitted in the latter part of the temporal window of integration (TWI) was longer than that of stimuli omitted in the first part of the TWI. The mean peak amplitude was found to be higher in the right hemisphere than in the frontal midline and left hemispheres in response to left ear stimulation. In conclusion, the results of this study suggest that would be incorrect to believe that the stimuli have strictly the characteristics of speech-sound. However. the results of the interaction effect in the latencies from omission were insignificant. These results suggest that the detection time for deviance may not be related to the stimulus ear. However, the type of deviant stimuli on latencies was found to be significant. This is because the detection of the deviants was delayed when a deviation occurred in the latter part of the TWI, regardless of the stimulation of the ear.

Predictive and perceptual phonemic processing in articulatory motor areas: A prediction potential & mismatch negativity study.

The recent finding of predictive brain signals preceding anticipated perceptual and linguistic stimuli opens new questions for experimental research. Here, we address the possible brain basis of phonological predictions regarding the features of specific speech sounds and their relationship to phonological priming. To this end, we recorded EEG correlates of both pre- and post-stimulus brain responses in a phonological priming study. Redundant spoken sounds induced stimulus expectations, which manifested as a slow-wave anticipatory activity (the Prediction Potential, PP), whereas articulatory-congruent (e.g.,/bƏ/in the context of expected/pƏ/) pairs elicited weaker post-stimulus MMN-like responses as compared to the articulatory-incongruent (e.g.,/bƏ/in the context of expected/dƏ/) pairs, a pattern reminiscent of perceptual priming mediated by articulatory-motor areas. Source analysis reveal clusters of activation in lateral prefrontal, temporal and ventral motor areas, thus providing the proof of the relevance of multimodal representation units subserving predictive and perceptual phonemic processing.

tDCS modulates speech perception and production in second language learners.

Accurate identification and pronunciation of nonnative speech sounds can be particularly challenging for adult language learners. The current study tested the effects of a brief musical training combined with transcranial direct current stimulation (tDCS) on speech perception and production in a second language (L2). The sample comprised 36 native Hebrew speakers, aged 18-38, who studied English as L2 in a formal setting and had little musical training. Training encompassed musical perception tasks with feedback (i.e., timbre, duration, and tonal memory) and concurrent tDCS applied over the left posterior auditory-related cortex (including posterior superior temporal gyrus and planum temporale). Participants were randomly assigned to anodal or sham stimulation. Musical perception, L2 speech perception (measured by a categorical AXB discrimination task) and speech production (measured by a speech imitation task) were tested before and after training. There were no tDCS-dependent effects on musical perception post-training. However, only participants who received active stimulation showed increased accuracy of L2 phoneme discrimination and greater change in the acoustic properties of L2 speech sound production (i.e., second formant frequency in vowels and center of gravity in consonants). The results of this study suggest neuromodulation can facilitate the processing of nonnative speech sounds in adult learners.

Effectiveness of an Ultrasound Visual Biofeedback Training for Tongue Shape Assessment During Speech Sound Production.

PURPOSE: This proof-of-concept study examined the effectiveness of an ultrasound visual biofeedback (UVB) training within the Participatory Adult Teaching Strategy framework for instructing speech-language pathologists (SLPs) on the assessment of sonographic tongue configuration for remediation of speech sound errors. METHOD: This research followed a multiple-baseline, multiple-probe single-case research methodology replicated across behaviors for data collection and analysis. Two school-based SPLs with no previous knowledge or experience with UVB were recruited. RESULTS: Visual analysis of data and effect size calculations using the percentage of nonoverlapping data indicated that training was highly effective in teaching SLPs skills for the assessment of sonographic tongue configuration and how to promote changes in tongue configuration for the remediation of speech sound errors. CONCLUSIONS: This study addressed the current gap in the literature regarding the lack of empirical evidence of UVB training for SLPs. Our findings support the exploration of evidence-based teaching strategies to train SLPs in the use of UVB for assessment of tongue configuration and remediation of speech sound errors. This can be of interest to academic programs and organizers of training opportunities through continued education units.

Drawing Talking: Listening to Children With Speech Sound Disorders.

PURPOSE: Listening to children using age-appropriate techniques supports evidence-based clinical decision-making. In this article, we test the Sound Effects Study Drawing Protocol, an arts-based technique, to support children with speech sound disorder (SSD) to express their views about talking. METHOD: Participants were 124 Australian children aged 4-5 years in the Sound Effects Study. Their parents and teachers were concerned about their talking, and they were assessed as having SSD on the Diagnostic Evaluation of Articulation and Phonology. Drawings and children's interpretations were elicited then analyzed using the (a) Who Am I? Draw-a-Person Scale and (b) Sound Effects Study Focal Points. RESULTS: Drawings were developmentally typical for 4- to 5-year-olds. The six Sound Effects Study Focal Points were identified across the 124 drawings: body parts and facial expressions, talking and listening, relationships and connection, positivity, negativity, and no talking. Participants portrayed talking and listening as an action requiring mouths and ears represented by symbols (letters, speech bubbles) or as an activity with a variety of people. Children typically portrayed themselves as happy when talking; however, some portrayed negativity and some chose not to draw talking. CONCLUSIONS: In keeping with Articles 12 and 13 of the Convention on the Rights of the Child, this research demonstrated that 4- to 5-year-old children with SSD can express their views about talking via drawing. Professionals may use the Sound Effects Study Drawing Protocol as a child-friendly technique to support children to express views to guide holistic, evidence-based, child-centered speech-language pathology practice.

Multi-modal imaging of the auditory-larynx motor network for voicing perception.

Voicing is one of the most important characteristics of phonetic speech sounds. Despite its importance, voicing perception mechanisms remain largely unknown. To explore auditory-motor networks associated with voicing perception, we firstly examined the brain regions that showed common activities for voicing production and perception using functional magnetic resonance imaging. Results indicated that the auditory and speech motor areas were activated with the operculum parietale 4 (OP4) during both voicing production and perception. Secondly, we used a magnetoencephalography and examined the dynamical functional connectivity of the auditory-motor networks during a perceptual categorization task of /da/-/ta/ continuum stimuli varying in voice onset time (VOT) from 0 to 40 ms in 10 ms steps. Significant functional connectivities from the auditory cortical regions to the larynx motor area via OP4 were observed only when perceiving the stimulus with VOT 30 ms. In addition, regional activity analysis showed that the neural representation of VOT in the auditory cortical regions was mostly correlated with categorical perception of voicing but did not reflect the perception of stimulus with VOT 30 ms. We suggest that the larynx motor area, which is considered to play a crucial role in voicing production, contributes to categorical perception of voicing by complementing the temporal processing in the auditory cortical regions.

Individuals With Congenital Amusia Show Degraded Speech Perception but Preserved Statistical Learning for Tone Languages.

PURPOSE: Individuals with congenital amusia exhibit degraded speech perception. This study examined whether adult Chinese Mandarin listeners with amusia were still able to extract the statistical regularities of Mandarin speech sounds, despite their degraded speech perception. METHOD: Using the gating paradigm with monosyllabic syllable-tone words, we tested 19 Mandarin-speaking amusics and 19 musically intact controls. Listeners heard increasingly longer fragments of the acoustic signal across eight duration-blocked gates. The stimuli varied in syllable token frequency and syllable-tone co-occurrence probability. The correct syllable-tone word, correct syllable-only, correct tone-only, and correct syllable-incorrect tone responses were compared respectively between the two groups using mixed-effects models. RESULTS: Amusics were less accurate than controls in terms of the correct word, correct syllable-only, and correct tone-only responses. Amusics, however, showed consistent patterns of top-down processing, as indicated by more accurate responses to high-frequency syllables, high-probability tones, and tone errors all in manners similar to those of the control listeners. CONCLUSIONS: Amusics are able to learn syllable and tone statistical regularities from the language input. This extends previous work by showing that amusics can track phonological segment and pitch cues despite their degraded speech perception. The observed speech deficits in amusics are therefore not due to an abnormal statistical learning mechanism. These results support rehabilitation programs aimed at improving amusics' sensitivity to pitch.

A mobile app that uses neurofeedback and multi-sensory learning methods improves reading abilities in dyslexia: A pilot study.

Reading comprehension is difficult to improve for children with dyslexia because of the continuing demands of orthographic decoding in combination with limited working memory capacity. Children with dyslexia get special education that improves spelling, phonemic and vocabulary awareness, however the latest research indicated that special education does not improve reading comprehension. With the aim of improving reading comprehension, reading speed and all other reading abilities of children with dyslexia, Auto Train Brain that is a novel mobile app using neurofeedback and multi-sensory learning methods was developed. With a clinical study, we wanted to demonstrate the effectiveness of Auto Train Brain on reading abilities. We compared the cognitive improvements obtained with Auto Train Brain with the improvements obtained with special dyslexia training. Auto Train Brain was applied to 16 children with dyslexia 60 times for 30 minutes. The control group consisted of 14 children with dyslexia who did not have remedial training with Auto Train Brain, but who did continue special education. The TILLS test was applied to both the experimental and the control group at the beginning of the experiment and after a 6-month duration from the first TILLS test. Comparison of the pre- and post- TILLS test results indicated that applying neurofeedback and multi-sensory learning method improved reading comprehension of the experimental group more than that of the control group statistically significantly. Both Auto Train Brain and special education improved phonemic awareness and nonword spelling.

Phonetic versus spatial processes during motor-oriented imitations of visuo-labial and visuo-lingual speech: A functional near-infrared spectroscopy study.

While a large amount of research has studied the facilitation of visual speech on auditory speech recognition, few have investigated the processing of visual speech gestures in motor-oriented tasks that focus on the spatial and motor features of the articulator actions instead of the phonetic features of auditory and visual speech. The current study examined the engagement of spatial and phonetic processing of visual speech in a motor-oriented speech imitation task. Functional near-infrared spectroscopy (fNIRS) was used to measure the haemodynamic activities related to spatial processing and audiovisual integration in the superior parietal lobe (SPL) and the posterior superior/middle temporal gyrus (pSTG/pMTG) respectively. In addition, visuo-labial and visuo-lingual speech were compared with examine the influence of visual familiarity and audiovisual association on the processes in question. fNIRS revealed significant activations in the SPL but found no supra-additive audiovisual activations in the pSTG/pMTG, suggesting that the processing of audiovisual speech stimuli was primarily focused on spatial processes related to action comprehension and preparation, whereas phonetic processes related to audiovisual integration was minimal. Comparisons between visuo-labial and visuo-lingual speech imitations revealed no significant difference in the activation of the SPL or the pSTG/pMTG, suggesting that a higher degree of visual familiarity and audiovisual association did not significantly influence how visuo-labial speech was processed compared with visuo-lingual speech. The current study offered insights on the pattern of visual-speech processing under a motor-oriented task objective and provided further evidence for the modulation of multimodal speech integration by voluntary selective attention and task objective.

Attention to speech: mapping distributed and selective attention systems.

When faced with situations where many people talk at once, individuals can employ different listening strategies to deal with the cacophony of speech sounds and to achieve different goals. In this fMRI study, we investigated how the pattern of neural activity is affected by the type of attention applied to speech in a simulated "cocktail party." Specifically, we compared brain activation patterns when listeners "attended selectively" to only one speaker and ignored all others, versus when they "distributed their attention" and followed several concurrent speakers. Conjunction analysis revealed a highly overlapping network of regions activated for both types of attention, including auditory association cortex (bilateral STG/STS) and frontoparietal regions related to speech processing and attention (bilateral IFG/insula, right MFG, left IPS). Activity within nodes of this network, though, was modulated by the type of attention required as well as the number of competing speakers. Auditory and speech-processing regions exhibited higher activity during distributed attention, whereas frontoparietal regions were activated more strongly during selective attention. These results suggest a common "attention to speech" network, which provides the computational infrastructure to deal effectively with multi-speaker input, but with sufficient flexibility to implement different prioritization strategies and to adapt to different listener goals.

Attention Differentially Affects Acoustic and Phonetic Feature Encoding in a Multispeaker Environment.

Humans have the remarkable ability to selectively focus on a single talker in the midst of other competing talkers. The neural mechanisms that underlie this phenomenon remain incompletely understood. In particular, there has been longstanding debate over whether attention operates at an early or late stage in the speech processing hierarchy. One way to better understand this is to examine how attention might differentially affect neurophysiological indices of hierarchical acoustic and linguistic speech representations. In this study, we do this by using encoding models to identify neural correlates of speech processing at various levels of representation. Specifically, we recorded EEG from fourteen human subjects (nine female and five male) during a "cocktail party" attention experiment. Model comparisons based on these data revealed phonetic feature processing for attended, but not unattended speech. Furthermore, we show that attention specifically enhances isolated indices of phonetic feature processing, but that such attention effects are not apparent for isolated measures of acoustic processing. These results provide new insights into the effects of attention on different prelexical representations of speech, insights that complement recent anatomic accounts of the hierarchical encoding of attended speech. Furthermore, our findings support the notion that, for attended speech, phonetic features are processed as a distinct stage, separate from the processing of the speech acoustics.SIGNIFICANCE STATEMENT Humans are very good at paying attention to one speaker in an environment with multiple speakers. However, the details of how attended and unattended speech are processed differently by the brain is not completely clear. Here, we explore how attention affects the processing of the acoustic sounds of speech as well as the mapping of those sounds onto categorical phonetic features. We find evidence of categorical phonetic feature processing for attended, but not unattended speech. Furthermore, we find evidence that categorical phonetic feature processing is enhanced by attention, but acoustic processing is not. These findings add an important new layer in our understanding of how the human brain solves the cocktail party problem.

How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing.

Both visual articulatory gestures and orthography provide information on the phonological content of speech. This EEG study investigated the integration between speech and these two visual inputs. A comparison of skilled readers' brain responses elicited by a spoken word presented alone versus synchronously with a static image of a viseme or a grapheme of the spoken word's onset showed that while neither visual input induced audiovisual integration on N1 acoustic component, both led to a supra-additive integration on P2, with a stronger integration between speech and graphemes on left-anterior electrodes. This pattern persisted in P350 time-window and generalized to all electrodes. The finding suggests a strong impact of spelling knowledge on phonetic processing and lexical access. It also indirectly indicates that the dynamic and predictive value present in natural lip movements but not in static visemes is particularly critical to the contribution of visual articulatory gestures to speech processing.

Phonetic Encoding Contributes to the Processing of Linguistic Prosody at the Word Level: Cross-Linguistic Evidence From Event-Related Potentials.

PURPOSE: This study aimed to examine whether abstract knowledge of word-level linguistic prosody is independent of or integrated with phonetic knowledge. METHOD: Event-related potential (ERP) responses were measured from 18 adult listeners while they listened to native and nonnative word-level prosody in speech and in nonspeech. The prosodic phonology (speech) conditions included disyllabic pseudowords spoken in Chinese and in English matched for syllabic structure, duration, and intensity. The prosodic acoustic (nonspeech) conditions were hummed versions of the speech stimuli, which eliminated the phonetic content while preserving the acoustic prosodic features. RESULTS: We observed language-specific effects on the ERP that native stimuli elicited larger late negative response (LNR) amplitude than nonnative stimuli in the prosodic phonology conditions. However, no such effect was observed in the phoneme-free prosodic acoustic control conditions. CONCLUSIONS: The results support the integration view that word-level linguistic prosody likely relies on the phonetic content where the acoustic cues embedded in. It remains to be examined whether the LNR may serve as a neural signature for language-specific processing of prosodic phonology beyond auditory processing of the critical acoustic cues at the suprasyllabic level.

Frequency-Following Responses to Speech Sounds Are Highly Conserved across Species and Contain Cortical Contributions.

Time-varying pitch is a vital cue for human speech perception. Neural processing of time-varying pitch has been extensively assayed using scalp-recorded frequency-following responses (FFRs), an electrophysiological signal thought to reflect integrated phase-locked neural ensemble activity from subcortical auditory areas. Emerging evidence increasingly points to a putative contribution of auditory cortical ensembles to the scalp-recorded FFRs. However, the properties of cortical FFRs and precise characterization of laminar sources are still unclear. Here we used direct human intracortical recordings as well as extracranial and intracranial recordings from macaques and guinea pigs to characterize the properties of cortical sources of FFRs to time-varying pitch patterns. We found robust FFRs in the auditory cortex across all species. We leveraged representational similarity analysis as a translational bridge to characterize similarities between the human and animal models. Laminar recordings in animal models showed FFRs emerging primarily from the thalamorecipient layers of the auditory cortex. FFRs arising from these cortical sources significantly contributed to the scalp-recorded FFRs via volume conduction. Our research paves the way for a wide array of studies to investigate the role of cortical FFRs in auditory perception and plasticity.

Selective auditory attention within naturalistic scenes modulates reactivity to speech sounds.

Rapid recognition and categorization of sounds are essential for humans and animals alike, both for understanding and reacting to our surroundings and for daily communication and social interaction. For humans, perception of speech sounds is of crucial importance. In real life, this task is complicated by the presence of a multitude of meaningful non-speech sounds. The present behavioural, magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) study was set out to address how attention to speech versus attention to natural non-speech sounds within complex auditory scenes influences cortical processing. The stimuli were superimpositions of spoken words and environmental sounds, with parametric variation of the speech-to-environmental sound intensity ratio. The participants' task was to detect a repetition in either the speech or the environmental sound. We found that specifically when participants attended to speech within the superimposed stimuli, higher speech-to-environmental sound ratios resulted in shorter sustained MEG responses and stronger BOLD fMRI signals especially in the left supratemporal auditory cortex and in improved behavioural performance. No such effects of speech-to-environmental sound ratio were observed when participants attended to the environmental sound part within the exact same stimuli. These findings suggest stronger saliency of speech compared with other meaningful sounds during processing of natural auditory scenes, likely linked to speech-specific top-down and bottom-up mechanisms activated during speech perception that are needed for tracking speech in real-life-like auditory environments.

Learning nonnative speech sounds changes local encoding in the adult human cortex.

Adults can learn to identify nonnative speech sounds with training, albeit with substantial variability in learning behavior. Increases in behavioral accuracy are associated with increased separability for sound representations in cortical speech areas. However, it remains unclear whether individual auditory neural populations all show the same types of changes with learning, or whether there are heterogeneous encoding patterns. Here, we used high-resolution direct neural recordings to examine local population response patterns, while native English listeners learned to recognize unfamiliar vocal pitch patterns in Mandarin Chinese tones. We found a distributed set of neural populations in bilateral superior temporal gyrus and ventrolateral frontal cortex, where the encoding of Mandarin tones changed throughout training as a function of trial-by-trial accuracy ("learning effect"), including both increases and decreases in the separability of tones. These populations were distinct from populations that showed changes as a function of exposure to the stimuli regardless of trial-by-trial accuracy. These learning effects were driven in part by more variable neural responses to repeated presentations of acoustically identical stimuli. Finally, learning effects could be predicted from speech-evoked activity even before training, suggesting that intrinsic properties of these populations make them amenable to behavior-related changes. Together, these results demonstrate that nonnative speech sound learning involves a wide array of changes in neural representations across a distributed set of brain regions.