Relationships Between Acoustic Characteristics and Intelligibility Scores: A Reanalysis of Japanese Speakers' Productions of American English Liquids.

The primary purpose of this research report was to investigate the relationships between acoustic characteristics and perceived intelligibility for native Japanese speakers' productions of American English liquids. This report was based on a reanalysis of intelligibility scores and acoustic analyses that were reported in two previous studies. We examined which acoustic parameters were associated with higher perceived intelligibility scores for their productions of /l/ and /ɹ/ in American English, and whether Japanese speakers' productions of the two liquids were acoustically differentiated from each other. Results demonstrated that the second formant (F2) was strongly correlated with the perceived intelligibility scores for the Japanese adults' productions. Results also demonstrated that the Japanese adults' and children's productions of /l/ and /ɹ/ were indeed differentiated by some acoustic parameters including the third formant (F3). In addition, some changes occurred in the Japanese children's productions over the course of 1 year. Overall, the present report shows that Japanese speakers of American English may be making a distinction between /l/ and /ɹ/ in production, although the distinctions are made in a different way compared with native English speakers' productions. These findings have implications for setting realistic goals for improving intelligibility of English /l/ and /ɹ/ for Japanese speakers, as well as theoretical advancement of second-language speech learning.

Talker familiarity and the accommodation of talker variability.

A fundamental problem in speech perception is how (or whether) listeners accommodate variability in the way talkers produce speech. One view of the way listeners cope with this variability is that talker differences are normalized - a mapping between talker-specific characteristics and phonetic categories is computed such that speech is recognized in the context of the talker's vocal characteristics. Consistent with this view, listeners process speech more slowly when the talker changes randomly than when the talker remains constant. An alternative view is that speech perception is based on talker-specific auditory exemplars in memory clustered around linguistic categories that allow talker-independent perception. Consistent with this view, listeners become more efficient at talker-specific phonetic processing after voice identification training. We asked whether phonetic efficiency would increase with talker familiarity by testing listeners with extremely familiar talkers (family members), newly familiar talkers (based on laboratory training), and unfamiliar talkers. We also asked whether familiarity would reduce the need for normalization. As predicted, phonetic efficiency (word recognition in noise) increased with familiarity (unfamiliar < trained-on < family). However, we observed a constant processing cost for talker changes even for pairs of family members. We discuss how normalization and exemplar theories might account for these results, and constraints the results impose on theoretical accounts of phonetic constancy.

An acoustic analysis of American English liquids by adults and children: Native English speakers and native Japanese speakers of English.

This study investigated acoustic characteristics of American English liquids produced by native English (NE) and native Japanese (NJ) speakers reported in Aoyama, Flege, Guion, Akahane-Yamada, and Yamada [(2004). J. Phonetics 32, 233-250]. For a larger longitudinal study, the data were collected twice to investigate the acquisition of American English by the NJ speakers (Time 1, Time 2). Aoyama, Flege, Guion, Akahane-Yamada, and Yamada [(2004). J. Phonetics 32, 233-250] evaluated productions of /l/ and /ɹ/ in the NE and NJ adults and children (16 participants each) using NE speakers' perceptual judgments and showed that the NJ children's production of /ɹ/ improved from Time 1 to Time 2. In the current study, four acoustic parameters (duration, F1, F2, and F3) were measured in 256 tokens each of English /l/ and /ɹ/. Results showed that some acoustic parameters, such as F2, changed from Time 1 to Time 2 in the NJ speakers' productions, indicating improvements. However, the NJ speakers' productions were different from the NE speakers' productions in almost all acoustic parameters at both Time 1 and Time 2. Results suggest that the improvements in the NJ children's productions of /ɹ/ reported in Aoyama, Flege, Guion, Akahane-Yamada, and Yamada [(2004). J. Phonetics 32, 233-250] were due to a combination of changes, not due to a change in one acoustic parameter such as F3 in /ɹ/.

Cross-language perceptual similarity predicts categorial discrimination of American vowels by naïve Japanese listeners.

Current speech perception models propose that relative perceptual difficulties with non-native segmental contrasts can be predicted from cross-language phonetic similarities. Japanese (J) listeners performed a categorical discrimination task in which nine contrasts (six adjacent height pairs, three front/back pairs) involving eight American (AE) vowels [iː, ɪ, ε, æː, ɑː, ʌ, ʊ, uː] in /hVbə/ disyllables were tested. The listeners also completed a perceptual assimilation task (categorization as J vowels with category goodness ratings). Perceptual assimilation patterns (quantified as categorization overlap scores) were highly predictive of discrimination accuracy (r(s)=0.93). Results suggested that J listeners used both spectral and temporal information in discriminating vowel contrasts.

A one-year longitudinal study of English and Japanese vowel production by Japanese adults and children in an English-speaking setting.

The effect of age of acquisition on first- and second-language vowel production was investigated. Eight English vowels were produced by Native Japanese (NJ) adults and children as well as by age-matched Native English (NE) adults and children. Productions were recorded shortly after the NJ participants' arrival in the USA and then one year later. In agreement with previous investigations [Aoyama, et al., J. Phon. 32, 233-250 (2004)], children were able to learn more, leading to higher accuracy than adults in a year's time. Based on the spectral quality and duration comparisons, NJ adults had more accurate production at Time 1, but showed no improvement over time. The NJ children's productions, however, showed significant differences from the NE children's for English "new" vowels /ɪ/, /ε/, /ɑ/, /ʌ/ and /ʊ/ at Time 1, but produced all eight vowels in a native-like manner at Time 2. An examination of NJ speakers' productions of Japanese /i/, /a/, /u/ over time revealed significant changes for the NJ Child Group only. Japanese /i/ and /a/ showed changes in production that can be related to second language (L2) learning. The results suggest that L2 vowel production is affected importantly by age of acquisition and that there is a dynamic interaction, whereby the first and second language vowels affect each other.

Training the brain to weight speech cues differently: a study of Finnish second-language users of English.

Foreign-language learning is a prime example of a task that entails perceptual learning. The correct comprehension of foreign-language speech requires the correct recognition of speech sounds. The most difficult speech-sound contrasts for foreign-language learners often are the ones that have multiple phonetic cues, especially if the cues are weighted differently in the foreign and native languages. The present study aimed to determine whether non-native-like cue weighting could be changed by using phonetic training. Before the training, we compared the use of spectral and duration cues of English /i/ and /I/ vowels (e.g., beat vs. bit) between native Finnish and English speakers. In Finnish, duration is used phonologically to separate short and long phonemes, and therefore Finns were expected to weight duration cues more than native English speakers. The cross-linguistic differences and training effects were investigated with behavioral and electrophysiological methods, in particular by measuring the MMN brain response that has been used to probe long-term memory representations for speech sounds. The behavioral results suggested that before the training, the Finns indeed relied more on duration in vowel recognition than the native English speakers did. After the training, however, the Finns were able to use the spectral cues of the vowels more reliably than before. Accordingly, the MMN brain responses revealed that the training had enhanced the Finns' ability to preattentively process the spectral cues of the English vowels. This suggests that as a result of training, plastic changes had occurred in the weighting of phonetic cues at early processing stages in the cortex.

Acoustic and perceptual similarity of Japanese and American English vowels.

Acoustic and perceptual similarities between Japanese and American English (AE) vowels were investigated in two studies. In study 1, a series of discriminant analyses were performed to determine acoustic similarities between Japanese and AE vowels, each spoken by four native male speakers using F1, F2, and vocalic duration as input parameters. In study 2, the Japanese vowels were presented to native AE listeners in a perceptual assimilation task, in which the listeners categorized each Japanese vowel token as most similar to an AE category and rated its goodness as an exemplar of the chosen AE category. Results showed that the majority of AE listeners assimilated all Japanese vowels into long AE categories, apparently ignoring temporal differences between 1- and 2-mora Japanese vowels. In addition, not all perceptual assimilation patterns reflected context-specific spectral similarity patterns established by discriminant analysis. It was hypothesized that this incongruity between acoustic and perceptual similarity may be due to differences in distributional characteristics of native and non-native vowel categories that affect the listeners' perceptual judgments.

Training English listeners to perceive phonemic length contrasts in Japanese.

The present study investigated the extent to which native English listeners' perception of Japanese length contrasts can be modified with perceptual training, and how their performance is affected by factors that influence segment duration, which is a primary correlate of Japanese length contrasts. Listeners were trained in a minimal-pair identification paradigm with feedback, using isolated words contrasting in vowel length, produced at a normal speaking rate. Experiment 1 tested listeners using stimuli varying in speaking rate, presentation context (in isolation versus embedded in carrier sentences), and type of length contrast. Experiment 2 examined whether performance varied by the position of the contrast within the word, and by whether the test talkers were professionally trained or not. Results did not show that trained listeners improved overall performance to a greater extent than untrained control participants. Training improved perception of trained contrast types, generalized to nonprofessional talkers' productions, and improved performance in difficult within-word positions. However, training did not enable listeners to cope with speaking rate variation, and did not generalize to untrained contrast types. These results suggest that perceptual training improves non-native listeners' perception of Japanese length contrasts only to a limited extent.

Neural processes involved with perception of non-native durational contrasts.

Neural processes underlying identification of durational contrasts were studied by comparing English and Japanese speakers for Japanese short/long vowel identification relative to consonant identification. Enhanced activities for non-native contrast (Japanese short/long vowel identification by English speakers) were observed in brain regions involved with articulatory-auditory mapping (Broca's area, superior temporal gyrus, planum temporale, and cerebellum), but not in the supramarginal gyrus. Greater activity in the supramarginal gyrus found for the consonant identification over short/long vowel identification by Japanese speakers implies that it is more important for phonetic contrasts differing in place of articulation than for vowel duration. These results support the hypothesis that neural processes used to facilitate perception depend on the relative contribution of information important for articulatory planning control.

An artificial environment is often a noisy environment: auditory scene analysis and speech perception in noise.

Our auditory system has to organize and to pick up a target sound with many components, sometimes rejecting irrelevant sound components, but sometimes forming multiple streams including the target stream. This situation is well described with the concept of auditory scene analysis. Research on speech perception in noise is closely related to auditory scene analysis. This paper briefly reviews the concept of auditory scene analysis and previous and ongoing research on speech perception in noise, and discusses the future direction of research. Further experimental investigations are needed to understand our perceptual mechanisms better.

Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory-auditory/orosensory internal models.

This experiment investigates neural processes underlying perceptual identification of the same phonemes for native- and second-language speakers. A model is proposed implicating the use of articulatory-auditory and articulatory-orosensory mappings to facilitate perceptual identification under conditions in which the phonetic contrast is ambiguous, as in the case of second-language speakers. In contrast, native-language speakers are predicted to use auditory-based phonetic representations to a greater extent for perceptual identification than second-language speakers. The English /r-l/ phonetic contrast, although easy for native English speakers, is extremely difficult for native Japanese speakers who learned English as a second language after childhood. Twenty-two native English and twenty-two native Japanese speakers participated in this study. While undergoing event-related fMRI, subjects were aurally presented with syllables starting with a /r/, /l/, or a vowel and were required to rapidly identify the phoneme perceived by pushing one of three buttons with the left thumb. Consistent with the proposed model, the results show greater activity for second- over native-language speakers during perceptual identification of /r/ and /l/ relative to vowels in brain regions implicated with instantiating forward and inverse articulatory-auditory articulatory-orosensory models [Broca's area, anterior insula, anterior superior temporal sulcus/gyrus (STS/G), planum temporale (PT), superior temporal parietal area (Stp), SMG, and cerebellum]. The results further show that activity in brain regions implicated with instantiating these internal models is correlated with better /r/ and /l/ identification performance for second-language speakers. Greater activity found for native-language speakers especially in the anterior STG/S for /r/ and /l/ perceptual identification is consistent with the hypothesis that native-language speakers use auditory phonetic representations more extensively than second-language speakers.

Learning-induced neural plasticity associated with improved identification performance after training of a difficult second-language phonetic contrast.

Adult native Japanese speakers have difficulty perceiving the English /r-l/ phonetic contrast even after years of exposure. However, after extensive perceptual identification training, long-lasting improvement in identification performance can be attained. This fMRI study investigates localized changes in brain activity associated with 1 month of extensive feedback-based perceptual identification training by native Japanese speakers learning the English /r-l/ phonetic contrast. Before and after training, separate functional brain imaging sessions were conducted for identification of the English /r-l/ contrast (difficult for Japanese speakers), /b-g/ contrast (easy), and /b-v/ contrast (difficult), in which signal-correlated noise served as the reference control condition. Neural plasticity, denoted by exclusive enhancement in brain activity for the /r-l/ contrast, does not involve only reorganization in brain regions concerned with acoustic-phonetic processing (superior and medial temporal areas) but also the recruitment of additional bilateral cortical (supramarginal gyrus, planum temporale, Broca's area, premotor cortex, supplementary motor area) and subcortical regions (cerebellum, basal ganglia, substantia nigra) involved with auditory-articulatory (perceptual-motor) mappings related to verbal speech processing and learning. Contrary to what one may expect, brain activity for perception of a difficult contrast does not come to resemble that of an easy contrast as learning proceeds. Rather, the results support the hypothesis that improved identification performance may be due to the acquisition of auditory-articulatory mappings allowing for perception to be made in reference to potential action.

Cortical evidence of the perceptual backward masking effect on /l/ and /r/ sounds from a following vowel in Japanese speakers.

We examined the influence of stimulus duration of foreign consonant vowel stimuli on the MMNm (magnetic counter part of mismatch negativity). In Experiment 1, /ra/ and /la/ stimuli were synthesized and subjects were native Japanese speakers who are known to have difficulty discriminating the stimuli. "Short" duration stimuli were terminated in the middle of the consonant-to-vowel transition (110 ms). They were nevertheless clearly identifiable by English speakers. A clear MMNm was observed only for short-duration stimuli but not for untruncated long-duration (150-ms) stimuli. We suggest that the diminished MMNm for longer duration stimuli result from more effective masking by the longer vowel part. In Experiment 2 we examined this hypothesis by presenting only the third formant (F3) component of the original stimuli, since the acoustic difference between /la/ and /ra/ is most evident in the third formant, whereas F1 and F2 play a major role in vowel perception. If the MMNm effect depends on the acoustic property of F3, a stimulus duration effect comparable to that found with the original /la/ and /ra/ stimuli might be expected. However, if the effect is attributable to the masking effect from the vowel, no influence of stimulus duration would be expected, since neither stimulus contains F1 and F2 components. In fact, the results showed that the "F3 only" stimuli did not show a duration effect; MMNm was always elicited independent of stimulus duration. The MMN stimulus duration effect is thus suggested to come from the backward masking of foreign consonants by subsequent vowels.

A perceptual interference account of acquisition difficulties for non-native phonemes.

This article presents an account of how early language experience can impede the acquisition of non-native phonemes during adulthood. The hypothesis is that early language experience alters relatively low-level perceptual processing, and that these changes interfere with the formation and adaptability of higher-level linguistic representations. Supporting data are presented from an experiment that tested the perception of English /r/ and /l/ by Japanese, German, and American adults. The underlying perceptual spaces for these phonemes were mapped using multidimensional scaling and compared to native-language categorization judgments. The results demonstrate that Japanese adults are most sensitive to an acoustic cue, F2, that is irrelevant to the English /r/-/l/ categorization. German adults, in contrast, have relatively high sensitivity to more critical acoustic cues. The results show how language-specific perceptual processing can alter the relative salience of within- and between-category acoustic variation, and thereby interfere with second language acquisition.