Perception of /r/ and /l/ in a stop cluster: evidence of cross-modal context effects.

Experiments were conducted investigating unimodal and cross-modal phonetic context effects on /r/ and /l/ identifications to test a hypothesis that context effects arise in early auditory speech processing. Experiment 1 demonstrated an influence of a preceding bilabial stop consonant on the acoustic realization of /r/ and /l/ produced within the stop clusters /ibri/ and /ibli/. In Experiment 2, members of an acoustic /iri/ to /ili/ continuum were paired with an acoustic /ibi/. These dichotic tokens were associated with an increase in "l" identification relative to the /iri/ to /ili/ continuum. In Experiment 3, the /iri/ to /ili/ tokens were dubbed onto a video of a talker saying /ibi/. This condition was associated with a reliable perceptual shift relative to an auditory-only condition in which the /iri/ to /ili/ tokens were presented by themselves, ruling out an account of these context effects as arising during early auditory processing.

Face and mouth inversion effects on visual and audiovisual speech perception.

Three experiments examined whether image manipulations known to disrupt face perception also disrupt visual speech perception. Research has shown that an upright face with an inverted mouth looks strikingly grotesque whereas an inverted face and an inverted face containing an upright mouth look relatively normal. The current study examined whether a similar sensitivity to upright facial context plays a role in visual speech perception. Visual and audiovisual syllable identification tasks were tested under 4 presentation conditions: upright face-upright mouth, inverted face-inverted mouth, inverted face-upright mouth, and upright face-inverted mouth. Results revealed that for some visual syllables only the upright face-inverted mouth image disrupted identification. These results suggest that upright facial context can play a role in visual speech perception. A follow-up experiment testing isolated mouths supported this conclusion.

The encoding of rate and talker information during phonetic perception.

The acoustic structure of the speech signal is extremely variable due to a variety of contextual factors, including talker characteristics and speaking rate. To account for the listener's ability to adjust to this variability, speech researchers have posited the existence of talker and rate normalization processes. The current study examined how the perceptual system encoded information about talker and speaking rate during phonetic perception. Experiments 1-3 examined this question, using a speeded classification paradigm developed by Garner (1974). The results of these experiments indicated that decisions about phonemic identity were affected by both talker and rate information: irrelevant variation in either dimension interfered with phonemic classification. While rate classification was also affected by phoneme variation, talker classification was not. Experiment 4 examined the impact of talker and rate variation on the voicing boundary under different blocking conditions. The results indicated that talker characteristics influenced the voicing boundary when talker variation occurred within a block of trials only under certain conditions. Rate variation, however, influenced the voicing boundary regardless of whether or not there was rate variation within a block of trials. The findings from these experiments indicate that phoneme and the rate information are encoded in an integral manner during speech perception, while talker characteristics are encoded separately.

Acoustic cues to place of articulation and the McGurk effect: the role of release bursts, aspiration, and formant transitions.

The McGurk effect demonstrates that the perceived place of articulation of an auditory consonant (such as/bi/) can be influenced by the simultaneous presentation of a videotape of a talker saying a conflicting consonant such as /gi/. Usually, such a presentation is perceived by observers as "di" or "delta i" (known as fusion responses). The reverse pairing (auditory /gi/ paired with a visual /bi/) results in "bgi." percepts. These are known as combination responses. In the current study, three experiments examined how acoustic information about place of articulation contained within the release bursts, aspiration, and voiced formants and transitions of a consonant contribute to the McGurk effect. In the first experiment, the release bursts and aspiration were deleted from the acoustic signal. This manipulation resulted in a smaller impact on McGurk "fusion" tokens relative to the McGurk "combination" tokens. This asymmetry may be related to the perceptual salience of the release bursts and aspiration for velar compared to the bilabial tokens used in this experiment and their importance for obtaining the combination percept. In Experiment 2, the release bursts and aspiration were increased in amplitude. Results revealed either no effect or a stronger McGurk effect for the manipulated tokens than for the intact tokens. This findings suggests that the McGurk effect for fusion tokens does not occur simply because the release bursts and aspiration are weak. In Experiment 3, low-pass filtering the second and higher formants and transitions was associated with the largest overall impact on the McGurk effect. This suggests that dynamic information contained within these formants is of primary importance in obtaining the McGurk effect. These cues are, however, context-dependent and vary as a function of talker and vowel context.

Spectral discontinuities and the vowel length effect.

Perception of voicing for stop consonants in consonant-vowel syllables can be affected by the duration of the following vowel so that longer vowels lead to more "voiced" responses. On the basis of several experiments, Green, Stevens, and Kuhl (1994) concluded that continuity of fundamental frequency (f0), but not continuity of formant structure, determined the effective length of the following vowel. In an extension of those efforts, we found here that both effects were critically dependent on particular f0s and formant values. First, discontinuity in f0 does not necessarily preclude the vowel length effect because the effect maintains when f0 changes from 200 to 100 Hz, and 200-Hz partials extend continuously through test syllables. Second, spectral discontinuity does preclude the vowel length effect when formant changes result in a spectral peak shifting to another harmonic. The results indicate that the effectiveness of stimulus changes for sustaining or diminishing the vowel length effect depends critically on particulars of spectral composition.

Cross-modal discrepancies in coarticulation and the integration of speech information: the McGurk effect with mismatched vowels.

Two experiments examined the impact of a discrepancy in vowel quality between the auditory and visual modalities on the perception of a syllable-initial consonant. One experiment examined the effect of such a discrepancy on the McGurk effect by cross-dubbing auditory /bi/ tokens onto visual /ga/ articulations (and vice versa). A discrepancy in vowel category significantly reduced the magnitude of the McGurk effect and changed the pattern of responses. A 2nd experiment investigated the effect of such a discrepancy on the speeded classification of the initial consonant. Mean reaction times to classify the tokens increased when the vowel information was discrepant between the 2 modalities but not when the vowel information was consistent. These experiments indicate that the perceptual system is sensitive to cross-modal discrepancies in the coarticulatory information between a consonant and its following vowel during phonetic perception.

Talker continuity and the use of rate information during phonetic perception.

Research has shown that speaking rate provides an important context for the perception of certain acoustic properties of speech. For example, syllable duration, which varies as a function of speaking rate, has been shown to influence the perception of voice onset time (VOT) for syllable-initial stop consonants. The purpose of the present experiments was to examine the influence of syllable duration when the initial portion of the syllable was produced by one talker and the remainder of the syllable was produced by a different talker. A short-duration and a long-duration /bi/-/pi/ continuum were synthesized with pitch and formant values appropriate to a female talker. When presented to listeners for identification, these stimuli demonstrated the typical effect of syllable duration on the voicing boundary: a shorter VOT boundary for the short stimuli than for the long stimuli. An /i/ vowel, synthesized with pitch and formant values appropriate to a male talker, was added to the end of each of the short tokens, producing a new hybrid continuum. Although the overall syllable duration of the hybrid stimuli equaled the original long stimuli, they produced a VOT boundary similar to that for the short stimuli. In a second experiment, two new /i/ vowels were synthesized. One had a pitch appropriate to a female talker with formant values appropriate to a male talker; the other had a pitch appropriate to a male talker and formants appropriate to a female talker. These vowels were used to create two new hybrid continua. In a third experiment, new hybrid continua were created by using more extreme male formant values. The results of both experiments demonstrated that the hybrid tokens with a change in pitch acted like the short stimuli, whereas the tokens with a change in formants acted like the long stimuli. A fourth experiment demonstrated that listeners could hear a change in talker with both sets of hybrid tokens. These results indicate that continuity of pitch but not formant structure appears to be the critical factor in the calculation of speaking rate within a syllable.

Integrating speech information across talkers, gender, and sensory modality: female faces and male voices in the McGurk effect.

Studies of the McGurk effect have shown that when discrepant phonetic information is delivered to the auditory and visual modalities, the information is combined into a new percept not originally presented to either modality. In typical experiments, the auditory and visual speech signals are generated by the same talker. The present experiment examined whether a discrepancy in the gender of the talker between the auditory and visual signals would influence the magnitude of the McGurk effect. A male talker's voice was dubbed onto a videotape containing a female talker's face, and vice versa. The gender-incongruent videotapes were compared with gender-congruent videotapes, in which a male talker's voice was dubbed onto a male face and a female talker's voice was dubbed onto a female face. Even though there was a clear incompatibility in talker characteristics between the auditory and visual signals on the incongruent videotapes, the resulting magnitude of the McGurk effect was not significantly different for the incongruent as opposed to the congruent videotapes. The results indicate that the mechanism for integrating speech information from the auditory and the visual modalities is not disrupted by a gender incompatibility even when it is perceptually apparent. The findings are compatible with the theoretical notion that information about voice characteristics of the talker is extracted and used to normalize the speech signal at an early stage of phonetic processing, prior to the integration of the auditory and the visual information.

Integral processing of visual place and auditory voicing information during phonetic perception.

Results of auditory speech experiments show that reaction times (RTs) for place classification in a test condition in which stimuli vary along the dimensions of both place and voicing are longer than RTs in a control condition in which stimuli vary only in place. Similar results are obtained when subjects are asked to classify the stimuli along the voicing dimension. By taking advantage of the "McGurk" effect (McGurk & MacDonald, 1976), the present study investigated whether a similar pattern of interference extends to situations in which variation along the place dimension occurs in the visual modality. The results showed that RTs for classifying phonetic features in the test condition were significantly longer than in the control condition for the place and voicing dimensions. These results indicate a mutual and symmetric interference exists in the classification of the two dimensions, even when the variation along the dimensions occurs in separate modalities.

Octamer and SPH motifs in the U1 enhancer cooperate to activate U1 RNA gene expression.

The transcriptional enhancer of a chicken U1 small nuclear RNA gene has been shown to extend over approximately 50 base pairs of DNA sequence located 180 to 230 base pairs upstream of the U1 transcription initiation site. It is composed of multiple functional motifs, including a GC box, an octamer motif, and a novel SPH motif. The contributions of these three distinct sequence motifs to enhancer function were studied with an oocyte expression assay. Under noncompetitive conditions in oocytes, the SPH motif is capable of stimulating U1 RNA transcription in the absence of the other functional motifs, whereas the octamer motif by itself lacks this ability. However, to form a transcription complex that is stable to challenge by a second competing small nuclear RNA transcription unit, both the octamer and SPH motifs are required. The GC box, although required for full enhancer activity, is not essential for stable complex formation in oocytes. Site-directed mutagenesis was used to study the DNA sequence requirements of the SPH motif. Functional activity of the SPH motif is spread throughout a 24-base-pair region 3' of the octamer but is particularly dependent upon sequences near an SphI restriction site located at the center of the SPH motif. Using embryonic chicken tissue as a source material, we identified and partially purified a factor, termed SBF, that binds sequence specifically to the SPH motif of the U1 enhancer. The ability of this factor to recognize and bind to mutant enhancer DNA fragments in vitro correlates with the functional activity of the corresponding enhancer sequences in vivo.

The role of visual information in the processing of place and manner features in speech perception.

Visual information provided by a talker's mouth movements can influence the perception of certain speech features. Thus, the "McGurk effect" shows that when the syllable (bi) is presented audibly, in synchrony with the syllable (gi), as it is presented visually, a person perceives the talker as saying (di). Moreover, studies have shown that interactions occur between place and voicing features in phonetic perception, when information is presented audibly. In our first experiment, we asked whether feature interactions occur when place information is specificed by a combination of auditory and visual information. Members of an auditory continuum ranging from (ibi) to (ipi) were paired with a video display of a talker saying (igi). The auditory tokens were heard as ranging from (ibi) to (ipi), but the auditory-visual tokens were perceived as ranging from (idi) to (iti). The results demonstrated that the voicing boundary for the auditory-visual tokens was located at a significantly longer VOT value than the voicing boundary for the auditory continuum presented without the visual information. These results demonstrate that place-voice interactions are not limited to situations in which place information is specified audibly.(ABSTRACT TRUNCATED AT 250 WORDS)