SHORT-TERM PERCEPTUAL TUNING TO TALKER CHARACTERISTICS.

When a listener encounters an unfamiliar talker, the ensuing perceptual accommodation to the unique characteristics of the talker has two aspects: (1) the listener assesses acoustic characteristics of speech to resolve the properties of the talker's sound production; and, (2) the listener appraises the talker's idiolect, subphonemic phonetic properties that compose the finest grain of linguistic production. A new study controlled a listener's exposure to determine whether the perceptual benefit rests on specific segmental experience. Effects of sentence exposure were measured using a spoken word identification task of Easy words (likely words drawn from sparse neighborhoods of less likely words) and Hard words (less likely words drawn from dense neighborhoods of more likely words). Recognition of words was facilitated by exposure to voiced obstruent consonants. Overall, these findings indicate that talker-specific perceptual tuning might depend more on exposure to phonemically marked consonants than to exposure distributed across the phoneme inventory.

PERCEPTUAL ORGANIZATION AND LAWFUL SPECIFICATION.

When a listener can also see a talker, audible and visible properties are ineluctably combined, perceptually. This perceptual disposition to audiovisual integration has received widely ranging explanations. At one extreme, accounts have likened perception to a blind listener and a deaf viewer combined within a single skin, resolving discrepancies in identification by each modality. At the other extreme, perception has been described as necessarily and automatically synesthetic. Useful descriptive and explanatory evidence was provided in a study of auditory-haptic presentation by Fowler and Dekle (1991), showing that neither familiarity nor congruence is required for perceptual integration to occur across modalities. Instead, the notion of conjoint lawful specification was proposed as a governing constraint. This principle treats sensory activity as proximal sampling of the properties of distal objects and events, and this essay notes that its corollaries offer a broadly applicable guide in contemporary investigations of perception.

Constraints on Sensitivity to Auditory Modulation in the Perceptual Organization of Speech.

BACKGROUND/STUDY CONTEXT: The perception of speech requires the integration of sensory details from a rapidly fading trace of a time-varying spectrum. This effortful cognitive function has been difficult to assess. New tests measuring intelligibility of sine-wave replicas of speech provided an assay of this critical function in normal-hearing young adults. METHODS: Four time-varying sinusoids replicated the frequency and amplitude variation of the natural resonances of spoken sentences. The temporal tolerance of perceptual integration of speech was measured by determining the effect on intelligibility of desynchronizing a single sine-wave component in each sentence. This method was applied in tests in which the sentences were temporally compressed or expanded over a 40% range. RESULTS: Desynchrony was harmful to perceptual integration over a narrow temporal range, indicating that modulation sensitivity is keyed to a rate of 20 Hz. No effect of variation in speech rate was observed on the intelligibility measure, whether rate was accelerated or decelerated relative to the natural rate. CONCLUSION: Performance measures of desynchrony tolerance did not vary when speech rate was accelerated or decelerated, revealing constraints on integration that are arguably primitive, sensory, auditory, and fixed. Because these are not adaptable, they limit the potential for perceptual learning in this aspect of perceptual organization. Implications for describing the elderly listener are noted.

Toddlers' comprehension of degraded signals: Noise-vocoded versus sine-wave analogs.

Recent findings suggest that development changes the ability to comprehend degraded speech. Preschool children showed greater difficulties perceiving noise-vocoded speech (a signal that integrates amplitude over broad frequency bands) than sine-wave speech (which maintains the spectral peaks without the spectrum envelope). In contrast, 27-month-old children in the present study could recognize speech with either type of degradation and performed slightly better with eight-channel vocoded speech than with sine-wave speech. This suggests that children's identification performance depends critically on the degree of degradation and that their success in recognizing unfamiliar speech encodings is encouraging overall.

ANALOGY AND DISANALOGY IN PRODUCTION AND PERCEPTION OF SPEECH.

A varied psychological vocabulary now describes the cognitive and social conditions of language production, the ultimate result of which is the mechanical action of vocal musculature in spoken expression. Following the logic of the speech chain, descriptions of production have often exhibited a clear analogy to accounts of perception. This reciprocality is especially evident in explanations that rely on reafference to control production, on articulation to inform perception, and on strict parity between produced and perceived form to provide invariance in the relation between abstract linguistic objects and observed expression. However, a causal account of production and perception cannot derive solely from this hopeful analogy. Despite sharing of abstract linguistic representations, the control functions in production and perception as well as the constraints on their use stand in fundamental disanalogy. This is readily seen in the different adaptive challenges to production - to speak in a single voice - and perception - to resolve familiar linguistic properties in any voice. This acknowledgment sets descriptive and theoretical challenges that break the symmetry of production and perception. As a consequence, this recognition dislodges an old impasse between the psychoacoustic and motoric accounts in the regulation of production and perception.

Modulation sensitivity in the perceptual organization of speech.

In a spoken utterance, a talker expresses linguistic constituents in serial order. A listener resolves these linguistic properties in the rapidly fading auditory sample. Classic measures agree that auditory integration occurs at a fine temporal grain. In contrast, recent studies have proposed that sensory integration of speech occurs at a coarser grain, approximate to the syllable, on the basis of indirect and relatively insensitive perceptual measures. Evidence from cognitive neuroscience and behavioral primatology has also been adduced to support the claim of sensory integration at the pace of syllables. In the present investigation, we used direct performance measures of integration, applying an acoustic technique to isolate the contribution of short-term acoustic properties to the assay of modulation sensitivity. In corroborating the classic finding of a fine temporal grain of integration, these functional measures can inform theory and speculation in accounts of speech perception.

Early recognition of speech.

Classic research on the perception of speech sought to identify minimal acoustic correlates of each consonant and vowel. In explaining perception, this view designated momentary components of an acoustic spectrum as cues to the recognition of elementary phonemes. This conceptualization of speech perception is untenable given the findings of phonetic sensitivity to modulation independent of the acoustic and auditory form of the carrier. The empirical key is provided by studies of the perceptual organization of speech, a low-level integrative function that finds and follows the sensory effects of speech amid concurrent events. These projects have shown that the perceptual organization of speech is keyed to modulation; fast; unlearned; nonsymbolic; indifferent to short-term auditory properties; and organization requires attention. The ineluctably multisensory nature of speech perception also imposes conditions that distinguish language among cognitive systems. WIREs Cogn Sci 2013, 4:213-223. doi: 10.1002/wcs.1213 This article is categorized under: Psychology > Language.

Estimating speech spectra for copy synthesis by linear prediction and by hand.

Linear prediction is a widely available technique for analyzing acoustic properties of speech, although this method is known to be error-prone. New tests assessed the adequacy of linear prediction estimates by using this method to derive synthesis parameters and testing the intelligibility of the synthetic speech that results. Matched sets of sine-wave sentences were created, one set using uncorrected linear prediction estimates of natural sentences, the other using estimates made by hand. Phoneme restrictions imposed on linguistic properties allowed comparisons between continuous and intermittent voicing, oral or nasal and fricative manner, and unrestricted phonemic variation. Intelligibility tests revealed uniformly good performance with sentences created by hand-estimation and a minimal decrease in intelligibility with estimation by linear prediction due to manner variation with continuous voicing. Poorer performance was observed when linear prediction estimates were used to produce synthetic versions of phonemically unrestricted sentences, but no similar decline was observed with synthetic sentences produced by hand estimation. The results show a substantial intelligibility cost of reliance on uncorrected linear prediction estimates when phonemic variation approaches natural incidence.

Auditory-phonetic projection and lexical structure in the recognition of sine-wave words.

Speech remains intelligible despite the elimination of canonical acoustic correlates of phonemes from the spectrum. A portion of this perceptual flexibility can be attributed to modulation sensitivity in the auditory-to-phonetic projection, although signal-independent properties of lexical neighborhoods also affect intelligibility in utterances composed of words. Three tests were conducted to estimate the effects of exposure to natural and sine-wave samples of speech in this kind of perceptual versatility. First, sine-wave versions of the easy and hard word sets were created, modeled on the speech samples of a single talker. The performance difference in recognition of easy and hard words was used to index the perceptual reliance on signal-independent properties of lexical contrasts. Second, several kinds of exposure produced familiarity with an aspect of sine-wave speech: (a) sine-wave sentences modeled on the same talker; (b) sine-wave sentences modeled on a different talker, to create familiarity with a sine-wave carrier; and (c) natural sentences spoken by the same talker, to create familiarity with the idiolect expressed in the sine-wave words. Recognition performance with both easy and hard sine-wave words improved after exposure only to sine-wave sentences modeled on the same talker. Third, a control test showed that signal-independent uncertainty is a plausible cause of differences in recognition of easy and hard sine-wave words. The conditions of beneficial exposure reveal the specificity of attention underlying versatility in speech perception.

Is desynchrony tolerance adaptable in the perceptual organization of speech?

Speech signal components that are desynchronized from the veridical temporal pattern lose intelligibility. In contrast, audiovisual presentations with large desynchrony in visible and audible speech streams are perceived without loss of integration. Under such conditions, the limit of desynchrony that permits audiovisual integration is also adaptable. A new project directly investigated the potential for adaptation to consistent desynchrony with unimodal auditory sine-wave speech. Listeners transcribed sentences that are highly intelligible, with veridical temporal properties. Desynchronized variants were created by leading or lagging the tone analog of the second formant relative to the rest of the tones composing the sentences, in 50-msec steps, ranging from 250-msec lead to 250-msec lag. In blocked trials, listeners only tolerated desynchronies <50 msec, and exhibited no gain in intelligibility to consistent desynchrony. Unimodal auditory and bimodal audiovisual forms of perceptual integration evidently exhibit different temporal characteristics, an indication of distinct perceptual functions.

AUDITORY-PHONETIC PROJECTION AND LEXICAL STRUCTURE IN THE RECOGNITION OF SINE-WAVE WORDS.

Speech remains intelligible despite the elimination of canonical acoustic correlates of phonemes from the spectrum. A portion of this perceptual flexibility can be attributed to modulation sensitivity in the auditory-to-phonetic projection, though signal-independent properties of lexical neighborhoods also affect intelligibility in utterances composed of words. Three tests were conducted to estimate the effects of exposure to natural and sine-wave samples of speech in this kind of perceptual versatility. First, sine-wave versions of the easy/hard word sets were created, modeled on the speech samples of a single talker. The performance difference in recognition of easy and hard words was used to index the perceptual reliance on signal-independent properties of lexical contrasts. Second, several kinds of exposure produced familiarity with an aspect of sine-wave speech: 1) sine-wave sentences modeled on the same talker; 2) sine-wave sentences modeled on a different talker, to create familiarity with a sine-wave carrier; and 3) natural sentences spoken by the same talker, to create familiarity with the idiolect expressed in the sine-wave words. Recognition performance with both easy and hard sine-wave words improved after exposure only to sine-wave sentences modeled on the same talker. Third, a control test showed that signal-independent uncertainty is a plausible cause of differences in recognition of easy and hard sine-wave words. The conditions of beneficial exposure reveal the specificity of attention underlying versatility in speech perception.

Asynchrony tolerance in the perceptual organization of speech.

Researchers have claimed that listeners tolerate large temporal distortion when integrating the spectral components of speech. In some estimates, perceivers resolve linguistic attributes at spectral desynchronies as great as the duration of a syllable. We obtained new measures of perceptual tolerance of auditory asynchrony, using sine-wave synthesis in order to require perceivers to resolve the speech stream dynamically. Listeners transcribed sentences in which the tone analogue of a second formant was desynchronized relative to the remaining tones of a sentence, with desynchrony ranging from a 250-msec lead to a 250-msec lag. Intelligibility declined symmetrically from 72% at synchrony to 7% at +/-100 msec. This finding of narrow asynchrony tolerance indicates a time-critical feature of the auditory perceptual organization of speech.

On the perception of similarity among talkers.

A listener who recognizes a talker notices characteristic attributes of the talker's speech despite the novelty of each utterance. Accounts of talker perception have often presumed that consistent aspects of an individual's speech, termed indexical properties, are ascribable to a talker's unique anatomy or consistent vocal posture distinct from acoustic correlates of phonetic contrasts. Accordingly, the perception of a talker is acknowledged to occur independently of the perception of a linguistic message. Alternatively, some studies suggest that attention to attributes of a talker includes indexical linguistic attributes conveyed in the articulation of consonants and vowels. This investigation sought direct evidence of attention to phonetic attributes of speech in perceiving talkers. Natural samples and sinewave replicas derived from them were used in three experiments assessing the perceptual properties of natural and sine-wave sentences; of temporally veridical and reversed natural and sine-wave sentences; and of an acoustic correlate of vocal tract scale to judgments of sine-wave talker similarity. The results revealed that the subjective similarity of individual talkers is preserved in the absence of natural vocal quality; and that local phonetic segmental attributes as well as global characteristics of speech can be exploited when listeners notice characteristics of talkers.

Short-term reorganization of auditory analysis induced by phonetic experience.

Sine wave replicas of spoken words can be perceived both as nonphonetic auditory forms and as words, depending on a listener's experience. In this study, brain areas activated by sine wave words were studied with fMRI in two conditions: when subjects perceived the sounds spontaneously as nonphonetic auditory forms ("nïve condition") and after instruction and brief practice attending to their phonetic attributes ("informed condition"). The test items were composed such that half replicated natural words ("phonetic items") and the other half did not, because the tone analogs of the first and third formants had been temporally reversed ("nonphonetic items"). Subjects were asked to decide whether an isolated tone analog of the second formant (T2) presented before the sine wave word (T1234) was included in it. Experience in attending to the phonetic properties of the sinusoids interfered with this auditory matching task and was accompanied by a decrease in auditory cortex activation with word replicas but not with the acoustically matched nonphonetic items. Because the activation patterns elicited by equivalent acoustic test items depended on a listener's awareness of their phonetic potential, this indicates that the analysis of speech sounds in the auditory cortex is distinct from the simple resolution of auditory form, and is not a mere consequence of acoustic complexity. Because arbitrary acoustic patterns did not evoke the response observed for phonetic patterns, these findings suggest that the perception of speech is contingent on the presence of familiar patterns of spectral variation. The results are consistent with a short-term functional reorganization of auditory analysis induced by phonetic experience with sine wave replicas and contingent on the dynamic acoustic structure of speech.

Analysis and analogy in the perception of vowels.

In two experiments, we investigated the creation of conceptual analogies to a contrast between vowels. An ordering procedure was used to determine the reliability of simple sensory and abstract analogies to vowel contrasts composed by naive volunteers. The results indicate that test subjects compose stable and consistent analogies to a meaningless segmental linguistic contrast, some invoking simple and complex relational properties. Although in the literature of psychophysics such facility has been explained as an effect of sensory analysis, the present studies indicate the action of a far subtler and more versatile cognitive function akin to the creation of meaning in figurative language.

Learning to recognize talkers from natural, sinewave, and reversed speech samples.

In 5 experiments, the authors investigated how listeners learn to recognize unfamiliar talkers and how experience with specific utterances generalizes to novel instances. Listeners were trained over several days to identify 10 talkers from natural, sinewave, or reversed speech sentences. The sinewave signals preserved phonetic and some suprasegmental properties while eliminating natural vocal quality. In contrast, the reversed speech signals preserved vocal quality while distorting temporally based phonetic properties. The training results indicate that listeners learned to identify talkers even from acoustic signals lacking natural vocal quality. Generalization performance varied across the different signals and depended on the salience of phonetic information. The results suggest similarities in the phonetic attributes underlying talker recognition and phonetic perception.

Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances.

Theoretical and practical motives alike have prompted recent investigations of multimodal speech perception. Theoretically, multimodal studies have extended the conceptualization of perceptual organization beyond the familiar modality-bound accounts deriving from Gestalt psychology. Practically, such investigations have been driven by a need to understand the proficiency of multimodal speech perception using an electrocochlear prosthesis for hearing. In each domain, studies have shown that perceptual organization of speech can occur even when the perceiver's auditory experience departs from natural speech qualities. Accordingly, our research examined auditor-visual multimodal integration of videotaped faces and selected acoustic constituents of speech signals, each realized as a single sinewave tone accompanying a video image of an articulating face. The single tone reproduced the frequency and amplitude of the phonatory cycle or of one of the lower three oral formants. Our results showed a distinct advantage for the condition pairing the video image of the face with a sinewave replicating the second formant, despite its unnatural timbre and its presentation in acoustic isolation from the rest of the speech signal. Perceptual coherence of multimodal speech in these circumstances is established when the two modalities concurrently specify the same underlying phonetic attributes.

On the perceptual organization of speech.

A general account of auditory perceptual organization has developed in the past 2 decades. It relies on primitive devices akin to the Gestalt principles of organization to assign sensory elements to probable groupings and invokes secondary schematic processes to confirm or to repair the possible organization. Although this conceptualization is intended to apply universally, the variety and arrangement of acoustic constituents of speech violate Gestalt principles at numerous junctures, cohering perceptually, nonetheless. The authors report 3 experiments on organization in phonetic perception, using sine wave synthesis to evade the Gestalt rules and the schematic processes alike. These findings falsify a general auditory account, showing that phonetic perceptual organization is achieved by specific sensitivity to the acoustic modulations characteristic of speech signals.