Direct neural coding of speech: Reconsideration of Whalen et al. (2006) (L).

Previous brain imaging results indicated that speech perception proceeded independently of the auditory primitives that are the product of primary auditory cortex [Whalen, Benson, Richardson, Swainson, Clark, Lai, Mencl, Fulbright, Constable, and Liberman (2006). J. Acoust. Soc. Am. 119, 575-581]. Recent evidence using electrocorticography [Hamilton, Oganian, Hall, and Chang (2021). Cell 184, 4626-4639] indicates that there is a more direct connection from subcortical regions to cortical speech regions than previous studies had shown. Although the mechanism differs, the Hamilton, Oganian, Hall, and Chang result supports the original conclusion even more strongly: Speech perception does not rely on the analysis of primitives from auditory analysis. Rather, the speech signal is processed as speech from the beginning.

The effect of native language and bilingualism on multimodal perception in speech: A study of audio-aerotactile integrationa).

Previous studies of speech perception revealed that tactile sensation can be integrated into the perception of stop consonants. It remains uncertain whether such multisensory integration can be shaped by linguistic experience, such as the listener's native language(s). This study investigates audio-aerotactile integration in phoneme perception for English and French monolinguals as well as English-French bilingual listeners. Six step voice onset time continua of alveolar (/da/-/ta/) and labial (/ba/-/pa/) stops constructed from both English and French end points were presented to listeners who performed a forced-choice identification task. Air puffs were synchronized to syllable onset and randomly applied to the back of the hand. Results show that stimuli with an air puff elicited more "voiceless" responses for the /da/-/ta/ continuum by both English and French listeners. This suggests that audio-aerotactile integration can occur even though the French listeners did not have an aspiration/non-aspiration contrast in their native language. Furthermore, bilingual speakers showed larger air puff effects compared to monolinguals in both languages, perhaps due to bilinguals' heightened receptiveness to multimodal information in speech.

Assessing accuracy of resonances obtained with reassigned spectrograms from the "ground truth" of physical vocal tract models.

The reassigned spectrogram (RS) has emerged as the most accurate way to infer vocal tract resonances from the acoustic signal [Shadle, Nam, and Whalen (2016). "Comparing measurement errors for formants in synthetic and natural vowels," J. Acoust. Soc. Am. 139(2), 713-727]. To date, validating its accuracy has depended on formant synthesis for ground truth values of these resonances. Synthesis is easily controlled, but it has many intrinsic assumptions that do not necessarily accurately realize the acoustics in the way that physical resonances would. Here, we show that physical models of the vocal tract with derivable resonance values allow a separate approach to the ground truth, with a different range of limitations. Our three-dimensional printed vocal tract models were excited by white noise, allowing an accurate determination of the resonance frequencies. Then, sources with a range of fundamental frequencies were implemented, allowing a direct assessment of whether RS avoided the systematic bias towards the nearest strong harmonic to which other analysis techniques are prone. RS was indeed accurate at fundamental frequencies up to 300 Hz; above that, accuracy was somewhat reduced. Future directions include testing mechanical models with the dimensions of children's vocal tracts and making RS more broadly useful by automating the detection of resonances.

Comparing metrics for quantification of children's tongue shape complexity using ultrasound imaging.

Speech sound disorders can pose a challenge to communication in children that may persist into adulthood. As some speech sounds are known to require differential control of anterior versus posterior regions of the tongue body, valid measurement of the degree of differentiation of a given tongue shape has the potential to shed light on development of motor skill in typical and disordered speakers. The current study sought to compare the success of multiple techniques in quantifying tongue shape complexity as an index of degree of lingual differentiation in child and adult speakers. Using a pre-existing data set of ultrasound images of tongue shapes from adult speakers producing a variety of phonemes, we compared the extent to which three metrics of tongue shape complexity differed across phonemes/phoneme classes that were expected to differ in articulatory complexity. We then repeated this process with ultrasound tongue shapes produced by a sample of young children. The results of these comparisons suggested that a modified curvature index and a metric representing the number of inflection points best reflected small changes in tongue shapes across individuals differing in vocal tract size. Ultimately, these metrics have the potential to reveal delays in motor skill in young children, which could inform assessment procedures and treatment decisions for children with speech delays and disorders.

Health effects of Indigenous language use and revitalization: a realist review.

BACKGROUND: Indigenous populations across the world are more likely to suffer from poor health outcomes when compared to other racial and ethnic groups. Although these disparities have many sources, one protective factor that has become increasingly apparent is the continued use and/or revitalization of traditional Indigenous lifeways: Indigenous language in particular. This realist review is aimed at bringing together the literature that addresses effects of language use and revitalization on mental and physical health. METHODS: Purposive bibliographic searches on Scopus were conducted to identify relevant publications, further augmented by forward citation chaining. Included publications (qualitative and quantitative) described health outcomes for groups of Indigenous people who either did or did not learn and/or use their ancestral language. The geographical area studied was restricted to the Americas, Australia or New Zealand. Publications that were not written in English, Spanish, French, Portuguese or German were excluded. A realist approach was followed to identify positive, neutral or negative effects of language use and/or acquisition on health, with both qualitative and quantitative measures considered. RESULTS: The bibliographic search yielded a total of 3508 possible publications of which 130 publications were included in the realist analysis. The largest proportion of the outcomes addressed in the studies (62.1%) reported positive effects. Neutral outcomes accounted for 16.6% of the reported effects. Negative effects (21.4%) were often qualified by such issues as possible cultural use of tobacco, testing educational outcomes in a student's second language, and correlation with socioeconomic status (SES), health access, or social determinants of health; it is of note that the positive correlations with language use just as frequently occurred with these issues as the negative correlations did. CONCLUSIONS: Language use and revitalization emerge as protective factors in the health of Indigenous populations. Benefits of language programs in tribal and other settings should be considered a cost-effective way of improving outcomes in multiple domains.

Formants are easy to measure; resonances, not so much: Lessons from Klatt (1986).

Formants in speech signals are easily identified, largely because formants are defined to be local maxima in the wideband sound spectrum. Sadly, this is not what is of most interest in analyzing speech; instead, resonances of the vocal tract are of interest, and they are much harder to measure. Klatt [(1986). in Proceedings of the Montreal Satellite Symposium on Speech Recognition, 12th International Congress on Acoustics, edited by P. Mermelstein (Canadian Acoustical Society, Montreal), pp. 5-7] showed that estimates of resonances are biased by harmonics while the human ear is not. Several analysis techniques placed the formant closer to a strong harmonic than to the center of the resonance. This "harmonic attraction" can persist with newer algorithms and in hand measurements, and systematic errors can persist even in large corpora. Research has shown that the reassigned spectrogram is less subject to these errors than linear predictive coding and similar measures, but it has not been satisfactorily automated, making its wider use unrealistic. Pending better techniques, the recommendations are (1) acknowledge limitations of current analyses regarding influence of F0 and limits on granularity, (2) report settings more fully, (3) justify settings chosen, and (4) examine the pattern of F0 vs F1 for possible harmonic bias.

Comparison of auto-contouring and hand-contouring of ultrasound images of the tongue surface.

Contours traced by trained phoneticians have been considered to be the most accurate way to identify the midsagittal tongue surface from ultrasound video frames. In this study, inter-measurer reliability was evaluated using measures that quantified both how closely human-placed contours approximated each other as well as how consistent measurers were in defining the start and end points of contours. High reliability across three measurers was found for all measures, consistent with treating contours placed by trained phoneticians as the 'gold standard.' However, due to the labour-intensive nature of hand-placing contours, automatic algorithms that detect the tongue surface are increasingly being used to extract tongue-surface data from ultrasound videos. Contours placed by six automatic algorithms (SLURP, EdgeTrak, EPCS, and three different configurations of the algorithm provided in Articulate Assistant Advanced) were compared to human-placed contours, with the same measures used to evaluate the consistency of the trained phoneticians. We found that contours defined by SLURP, EdgeTrak, and two of the AAA configurations closely matched the hand-placed contours along sections of the image where the algorithms and humans agreed that there was a discernible contour. All of the algorithms were much less reliable than humans in determining the anterior (tongue-tip) edge of tongue contours. Overall, the contours produced by SLURP, EdgeTrak, and AAA should be useable in a variety of clinical applications, subject to spot-checking. Additional practical considerations of these algorithms are also discussed.

Phonetic Documentation in Three Collections: Topics and Evolution.

Phonetic aspects of many languages have been documented, though the breadth and focus of such documentation varies substantially. In this survey, phonetic aspects (here called "categories") that are typically reported were assessed in three English-language collections-the Illustrations of the IPA, articles from the Journal of Phonetics, and papers from the Ladefoged/Maddieson Sounds of the World's Languages (SOWL) documentation project. Categories were defined for consonants (e.g., Voice Onset Time (VOT) and frication spectrum; 10 in total), vowels (e.g., formants and duration; 7 total) and suprasegmentals (e.g., stress and distinctive vowel length, 6 total). The Illustrations, due to their brevity, had, on average, limited coverage of the selected categories (12% of the 23 categories). Journal of Phonetics articles were typically theoretically motivated, but 64 had sufficient measurements to count as phonetic documentation; these also covered 12% of the categories. The SOWL studies, designed to cover as much of the phonetic structure as feasible in an article-length treatment, achieved 41% coverage on average. Four book-length studies were also examined, with an average of 49% coverage. Phonetic properties of many language families have been studied, though Indo-European is still disproportionately represented. Physiological measures were excluded as being less common, and perceptual measures were excluded as being typically more theoretical. This preliminary study indicates that certain acoustic properties of languages are typically measured and may be considered as an impetus for later, fuller coverage, but broader consensus on the categories is needed. Current documentation efforts could be more useful if these considerations were addressed.

Effects of Ultrasound Familiarization on Production and Perception of Nonnative Contrasts.

BACKGROUND/AIMS: We investigated the efficacy of ultrasound imaging of the tongue as a tool for familiarizing naïve learners with the production of a class of nonnative speech sounds: palatalized Russian consonants. METHODS: Two learner groups were familiarized, one with ultrasound and one with audio only. Learners performed pre- and postfamiliarization production and discrimination tasks. RESULTS: Ratings of productions of word-final palatalized consonants by learners from both groups improved after familiarization, as did discrimination of the palatalization contrast word-finally. There were no significant differences in the improvement between groups in either task. All learners were able to generalize to novel contexts in production and discrimination. The presence of palatalization interfered with discrimination of word-initial manner, and ultrasound learners were more successful in overcoming that interference. CONCLUSION: Ultrasound familiarization resulted in improvements in production and discrimination comparable to audio only. Ultrasound familiarization additionally helped learners overcome difficulties in manner discrimination introduced by palatalization. When familiarizing learners with a novel, nonnative class of sounds, a small set of stimuli in different contexts may be more beneficial than using a larger set in one context. Although untrained production can disrupt discrimination training, we found that production familiarization was not disruptive to discrimination or production.

F0-induced formant measurement errors result in biased variabilities.

Many developmental studies attribute reduction of acoustic variability to increasing motor control. However, linear prediction-based formant measurements are known to be biased toward the nearest harmonic of F0, especially at high F0s. Thus, the amount of reported formant variability generated by changes in F0 is unknown. Here, 470 000 vowels were synthesized, mimicking statistics reported in four developmental studies, to estimate the proportion of formant variability that can be attributed to F0 bias, as well as other formant measurement errors. Results showed that the F0-induced formant measurements errors are large and systematic, and cannot be eliminated by a large sample size.

Comparing measurement errors for formants in synthetic and natural vowels.

The measurement of formant frequencies of vowels is among the most common measurements in speech studies, but measurements are known to be biased by the particular fundamental frequency (F0) exciting the formants. Approaches to reducing the errors were assessed in two experiments. In the first, synthetic vowels were constructed with five different first formant (F1) values and nine different F0 values; formant bandwidths, and higher formant frequencies, were constant. Input formant values were compared to manual measurements and automatic measures using the linear prediction coding-Burg algorithm, linear prediction closed-phase covariance, the weighted linear prediction-attenuated main excitation (WLP-AME) algorithm [Alku, Pohjalainen, Vainio, Laukkanen, and Story (2013). J. Acoust. Soc. Am. 134(2), 1295-1313], spectra smoothed cepstrally and by averaging repeated discrete Fourier transforms. Formants were also measured manually from pruned reassigned spectrograms (RSs) [Fulop (2011). Speech Spectrum Analysis (Springer, Berlin)]. All but WLP-AME and RS had large errors in the direction of the strongest harmonic; the smallest errors occur with WLP-AME and RS. In the second experiment, these methods were used on vowels in isolated words spoken by four speakers. Results for the natural speech show that F0 bias affects all automatic methods, including WLP-AME; only the formants measured manually from RS appeared to be accurate. In addition, RS coped better with weaker formants and glottal fry.

Indexing head movement during speech production using optical markers.

Optical marker tracking integrated with electromagnetic articulometry was used to assess the movement extent of various points on (a) forehead skin and (b) points on a head-mounted apparatus, relative to a fixed point just above the upper incisors, and to compare the accuracy of the two different approaches to indexing head position during speech production. Both methods can provide a satisfactory index of head position. If skin-affixed markers are used, a minimum of 4 is recommended. Locations for optimal marker placement are identified.

Methods for quantifying tongue shape and complexity using ultrasound imaging.

Quantification of tongue shape is potentially useful for indexing articulatory strategies arising from intervention, therapy and development. Tongue shape complexity is a parameter that can be used to reflect regional functional independence of the tongue musculature. This paper considers three different shape quantification methods - based on Procrustes analysis, curvature inflections and Fourier coefficients - and uses a linear discriminant analysis to test how well each method is able to classify tongue shapes from different phonemes. Test data are taken from six native speakers of American English producing 15 phoneme types. Results classify tongue shapes accurately when combined across quantification methods. These methods hold promise for extending the use of ultrasound in clinical assessments of speech deficits.

Production of the English /ɹ/ by Mandarin-English Bilingual Speakers.

Rhotic sounds are some of the most challenging sounds for L2 learners to acquire. This study investigates the production of English rhotic sounds by Mandarin-English bilinguals with two English proficiency levels. The production of the English /ɹ/ by 17 Mandarin-English bilinguals was examined with ultrasound imaging and compared with the production of native English speakers. The ultrasound data show that bilinguals can produce native-like bunched and retroflex gestures, but the distributional pattern of tongue shapes in various contexts differs from that of native speakers. Acoustically, the English /ɹ/ produced by bilinguals had a higher F3 and F3-F2, as well as some frication noise in prevocalic /ɹ/, features similar to the Mandarin /ɹ/. Mandarin-English bilinguals did produce language-specific phonetic realizations for the English and Mandarin /ɹ/s. There was a positive correlation between language proficiency and English-specific characteristics of /ɹ/ by Mandarin-English bilinguals in both articulation and acoustics. Phonetic similarities facilitated rather than hindered L2 speech learning in production: Mandarin-English bilinguals showed better performance in producing the English /ɹ/ allophones that were more similar to the Mandarin /ɹ/ (syllabic and postvocalic /ɹ/s) than producing the English /ɹ/ allophone that was less similar to the Mandarin /ɹ/ (prevocalic /ɹ/). This study contributes to our understanding of the mechanism of speech production in late bilinguals.

Hearing tongue loops: perceptual sensitivity to acoustic signatures of articulatory dynamics.

Previous work has shown that velar stops are produced with a forward movement during closure, forming a forward (anterior) loop for a VCV sequence, when the preceding vowels are back or mid. Are listeners aware of this aspect of articulatory dynamics? The current study used articulatory synthesis to examine how such kinematic patterns are reflected in the acoustics, and whether those acoustic patterns elicit different goodness ratings. In Experiment I, the size and direction of loops was modulated in articulatory synthesis. The resulting stimuli were presented to listeners for a naturalness judgment. Results show that listeners rate forward loops as more natural than backward loops, in agreement with typical productions. Acoustic analysis of the synthetic stimuli shows that forward loops exhibit shorter and shallower VC transitions than CV transitions. In Experiment II, three acoustic parameters were employed incorporating F3-F2 distance, transition slope, and transition length to systematically modulate the magnitude of VC and CV transitions. Listeners rated the naturalness in accord with those of Experiment I. This study reveals that there is sufficient information in the acoustic signature of "velar loops" to affect perceptual preference. Similarity to typical productions seemed to determine preferences, not acoustic distinctiveness.

The coarticulation/invariance scale: mutual information as a measure of coarticulation resistance, motor synergy, and articulatory invariance.

Coarticulation and invariance are two topics at the center of theorizing about speech production and speech perception. In this paper, a quantitative scale is proposed that places coarticulation and invariance at the two ends of the scale. This scale is based on physical information flow in the articulatory signal, and uses Information Theory, especially the concept of mutual information, to quantify these central concepts of speech research. Mutual Information measures the amount of physical information shared across phonological units. In the proposed quantitative scale, coarticulation corresponds to greater and invariance to lesser information sharing. The measurement scale is tested by data from three languages: German, Catalan, and English. The relation between the proposed scale and several existing theories of coarticulation is discussed, and implications for existing theories of speech production and perception are presented.

Perceptual Integration of Visual Evidence of the Airstream from Aspirated Stops.

This study investigates whether indirect visual evidence of aspiration can influence speech perception as previously found for tactile information. Participants were shown video of a speaker producing the sequence "pom" and "bomb" in a noisy setting. In some tokens, a candle was visibly perturbed by aspiration. All participants were more likely to correctly identify "pom" and incorrectly identify "bomb" in the presence of visible perturbation, indicating that perceptual integration was taking place. This effect was stronger for participants who reported being consciously aware of the candle as a predictor. This indicates that ambient information can be incorporated in speech perception even when presented via an indirect modality, and that active attention can amplify this effect.

Neural Changes in Children With Residual Speech Sound Disorder After Ultrasound Biofeedback Speech Therapy.

PURPOSE: Children with residual speech sound disorders (RSSD) have shown differences in neural function for speech production, as compared to their typical peers; however, information about how these differences may change over time and relative to speech therapy is needed. To address this gap, we used functional magnetic resonance imaging (fMRI) to examine functional activation and connectivity on adaptations of the syllable repetition task (SRT-Early Sounds and SRT-Late Sounds) in children with RSSD before and after a speech therapy program. METHOD: Sixteen children with RSSD completed an fMRI experiment before (Time 1) and after (Time 2) a speech therapy program with ultrasound visual feedback for /ɹ/ misarticulation. Progress in therapy was measured via perceptual ratings of productions of untreated /ɹ/ word probes. To control for practice effects and developmental change in patterns of activation and connectivity, 17 children with typical speech development (TD) completed the fMRI at Time 1 and Time 2. Functional activation was analyzed using a region-of-interest approach and functional connectivity was analyzed using a seed-to-voxel approach. RESULTS: Children with RSSD showed a range of responses to therapy. After correcting for multiple comparisons, we did not observe any statistically significant cross-sectional differences or longitudinal changes in functional activation. A negative relationship between therapy effect size and functional activation in the left visual association cortex was on the SRT-Late Sounds after therapy, but it did not survive correction for multiple comparisons. Significant longitudinal changes in functional connectivity were observed for the RSSD group on SRT-Early Sounds and SRT-Late Sounds, as well as for the TD group on the SRT-Early Sounds. RSSD and TD groups showed connectivity differences near the left insula on the SRT-Late Sounds at Time 2. CONCLUSION: RSSD and treatment with ultrasound visual feedback may thus be associated with neural differences in speech motor and visual association processes recruited for speech production.

Biomechanically preferred consonant-vowel combinations fail to appear in adult spoken corpora.

Certain consonant/vowel (CV) combinations are more frequent than would be expected from the individual C and V frequencies alone, both in babbling and, to a lesser extent, in adult language, based on dictionary counts: Labial consonants co-occur with central vowels more often than chance would dictate; coronals co-occur with front vowels, and velars with back vowels (Davis & MacNeilage, 1994). Plausible biomechanical explanations have been proposed, but it is also possible that infants are mirroring the frequency of the CVs that they hear. As noted, previous assessments of adult language were based on dictionaries; these "type" counts are incommensurate with the babbling measures, which are necessarily "token" counts. We analyzed the tokens in two spoken corpora for English, two for French and one for Mandarin. We found that the adult spoken CV preferences correlated with the type counts for Mandarin and French, not for English. Correlations between the adult spoken corpora and the babbling results had all three possible outcomes: significantly positive (French), uncorrelated (Mandarin), and significantly negative (English). There were no correlations of the dictionary data with the babbling results when we consider all nine combinations of consonants and vowels. The results indicate that spoken frequencies of CV combinations can differ from dictionary (type) counts and that the CV preferences apparent in babbling are biomechanically driven and can ignore the frequencies of CVs in the ambient spoken language.