Probing the cognitive representation of musical time: structural constraints on the perception of timing perturbations.

To determine whether structural factors interact with the perception of musical time, musically literate listeners were presented repeatedly with eight-bar musical excerpts, realized with physically regular timing on an electronic piano. On each trial, one or two randomly chosen time intervals were lengthened by a small amount, and the score. The resulting detection accuracy profile across all positions in each musical excerpt showed pronounced dips in places where lengthening would typically occur in an expressive (temporally modulated) performance. False alarm percentages indicated that certain tones seemed longer a priori, and these were among the ones whose actual lengthening was easiest to detect. The detection accuracy and false alarm profiles were significantly correlated with each other and with the temporal microstructure of expert performances, as measured from sound recordings by famous artists. Thus the detection task apparently tapped into listeners' musical thought and revealed their expectations about the temporal microstructure of music performance. These expectations, like the timing patterns of actual performances, derive from the cognitive representation of musical structure, as cued by a variety of systemic factors (grouping, meter, harmonic progression) and their acoustic correlates. No simple psycho-acoustic explanation of the detection accuracy profiles was evident. The results suggest that the perception of musical time is not veridical but "warped" by the structural representation. This warping may provide a natural basis for performance evaluation: expected timing patterns sound more or less regular, unexpected ones irregular. Parallels to language performance and perception are noted.

Perceptual coherence of speech: stability of silence-cued stop consonants.

A series of experiments was conducted to examine the perceptual stability of stop consonants cued by silence alone, as when [s] + silence + [laet] is perceived as splat. Following a replication of this perceptual integration phenomenon (Experiment 1), attempts were made to block it by instructing subjects to disregard the initial [s] and to focus instead on the onset of the following signal, which was varied from [plaet] to [laet]. However, these instructions had little effect at short silence durations (Experiment 2), and they reduced stop percepts for only 2 subjects at longer silence durations (Experiment 3). That is, subjects were generally unable to voluntarily dissociate the [s] noise from the following signal and thus to perceive the silent interval as silence rather than as a carrier of phonetic information. A low-uncertainty paradigm facilitated the task somewhat (Experiment 4). However, when the [s] frication was replaced with broadband noise (Experiment 5), listeners had no trouble at all in the selective-attention task, except at very short silence durations (less than 40 ms). This last finding suggests that, except for the shortest durations, the effect of silence on phonetic perception does not arise at the level of psychoacoustic stimulus interactions. Rather, the results support the hypothesis that perceptual integration of speech components, including silence, is a largely obligatory perceptual function driven by the listener's tacit knowledge of phonetic regularities.

Variations on a theme by chopin: relations between perception and production of timing in music.

A note interonset interval (IOI) increment in mechanically timed music is more difficult to detect where expressive lengthening typically occurs in artistic performance. Experiment 1 showed this in an excerpt from a Chopin etude and extended the task to IOI decrement detection. A simple measure of variation in perceptual bias was derived that correlated highly with the average timing pattern of pianists' performances, more so than with acoustic surface properties of the music. Similar results, but decreasing correlations, were obtained in each of four subsequent experiments in which the music was simplified in stages. Although local psychoacoustic effects on time perception cannot be ruled out completely, the results suggest that musical structure (melodic-rhythmic grouping in particular) has temporal implications that are reflected not only in musicians' motor behavior but also in listeners' time-keeping abilities.

Phase correction, phase resetting, and phase shifts after subliminal timing perturbations in sensorimotor synchronization.

Recent studies of synchronized finger tapping have shown that perceptually subliminal phase shifts in an auditory sequence are rapidly compensated for in the motor activity (B. H. Repp, 2000a). Experiment 1 used a continuation-tapping task to confirm that this compensation is indeed a phase correction, not an adjustment of the central timekeeper period. Experiments 2-5 revealed that this phase correction occurs even when there is no ordinary sensorimotor asynchrony--when the finger taps are in antiphase or arbitrary phase relative to the auditory sequence (Experiments 2 and 3) or when the tap coinciding with the sequence phase shift is withheld (Experiments 4 and 5). The phase correction observed in the latter conditions was instantaneous, which suggests that phase resetting occurs when the motor activity is discontinuous. A prolonged phase shift suggestive of overcompensation was observed in some conditions, which poses a challenge to pure phase correction models.

Effects of preceding context on the voice-onset-time category boundary.

The category boundary on a voice-onset-time (VOT) continuum ranging from bin to pin shifts when the stimuli are preceded by different carrier phrases. By using a variety of precursor phrases, by varying the temporal interval between precursor and test word, and by selectively eliminating either voicing information or spectral structure from the precursors, the present experiments show that the context effect is caused by the presence or absence of voicing in the precursor's final segment. The effect decreases with temporal separation but persists over several seconds. In addition, the "baseline" VOT boundary for isolated test words interspersed in a test sequence shifts depending on what precursor stimuli occur in the same sequence. The perception of VOT thus seems to be sensitive to both close and distant manifestations of laryngeal activity, always in an assimilative fashion, which suggests an integrative perceptual mechanism with a long time constant.

Context independence and phonetic mediation in categorical perception.

Categorical perception, as an ideal situation, is rarely if ever observed in the laboratory. Two separate requirements must be met for categorical perception: (a) Discrimination performance must be phonetically mediated (i.e., it should be predictable from labeling performance), (b) labeling responses must be independent of the stimulus context. To determine the extent to which departures from the ideal are due to failures to meet one or both of these requirements, we used four stimulus continua in same-different (AX) discrimination and labeling tasks: stop-consonant-vowel (CV) syllables, isolated vowels, isolated fricative noises, and nonspeech sounds varying in timbre. We found that perception of CV syllables fell short of the ideal only because of contextual influences on labeling. Neither criterion of categorical perception was met by vowels or fricative noises, but subjects were more prone to phonetic mediation with vowels than with fricative noises, and they showed more context independence with fricative noises than with vowels. Surprisingly, the nonspeech timbre stimuli satisfied both requirements better than either vowels or fricative noises. This findings was attributed to the short duration of our timbre stimuli, which prevented them from sounding vowel-like but at the same time may have prevented stable auditory memory traces.

Perceptual integration of acoustic cues for stop, fricative, and affricate manner.

Introducing a short interval of silence between the words SAY and SHOP causes listeners to hear SAY CHOP. Another cue for the fricative-affricate distinction is the duration of the fricative noise in SHOP (CHOP). Now, varying both these temporal cues orthogonally in a sentence context, we find that, within limits, they are perceived in relation to each other: The shorter the duration of the noise, the shorter the silence necessary to convert the fricative into an affricate. On the other hand, when the rate of articulation of the sentence frame is increased while holding noise duration constant, a longer silent interval is needed to hear an affricate, as if the noise duration, but not the silence duration, were effectively longer in the faster sentence. In a second experiment, varying noise and silence durations in GRAY SHIP, we find that given sufficient silence, listeners report GRAY CHIP when the noise is short but GREAT SHIP when it is long. Thus, the long noise in the second syllable disposes listeners to displace the stop to the first syllable, so that they hear not a syllable-initial affricate (i.e., stop-initiated fricative) but a syllable-final stop (followed by a syllable-initial fricative). Repeating the experiment with GREAT SHIP as the original utterance, we obtain the same pattern of results, together with only a moderate increase in GREAT responses. In all such cases, the listeners integrate a numerous, diverse, and temporally distributed set of acoustic cues into a unitary phonetic percept. These several cues have in common only that they are the products of a unitary articulatory act. In effect, then, it is the articulatory act that is perceived.

Categories and context in the perception of isolated steady-state vowels.

The noncategorical perception of isolated vowels has been attributed to the availability of auditory memory in discrimination. In our first experiment, using vowels from an /i/-/I/epsilon) continuum in a same-different (AX) task and comparing the results with predictions derived from a separate identification test, we demonstrated that vowels are perceived more nearly categorically if auditory memory is degraded by extending the interstimulus interval and/or filling it with irrelevant vowel sounds. In a second experiment, we used a similar paradigm, but in addition to presenting a separate identification test, we elicited labeling responses to the AX pairs used in the discrimination task. We found that AX labeling responses predicted discrimination performance quite well, regardless of whether auditory memory was available, whereas the predictions from the separate identification test were more poorly matched by the obtained data. The AX labeling reponses showed large contrast effects (both proactive and retroactive) that were greatly reduced when auditory memory was interfered with. We conclude from the presence of these contrast effects that vowels are not perceived categorically (that is, absolutely). However, it seems that by taking the effects of context into account properly, discrimination performance can be quite accurately predicted from labeling data, suggesting that vowel discrimination, like consonant discrimination, may be mediated by phonetic labels.

Processes underlying adaptation to tempo changes in sensorimotor synchronization.

In synchronizing finger taps with an auditory sequence, a small sudden tempo ("step") change in the sequence tends to be followed by rapid adaptation of the tapping period but slow adaptation of the relative phase of the taps, whereas a larger step change leads to initial period overshoot followed by rapid adaptation of both period and phase [M.H. Thaut, R.A. Miller, L.M. Schauer, Biological Cybernetics 79 (1998a) 241-250]. Experiment 1 replicated these findings and showed that the transition between the two patterns of adaptation occurs near the perceptual detection threshold for a tempo change. A reasonable explanation of these data was provided by a dual-process model of internal error correction [J. Mates, Biological Cybernetics 70 (1994a) 463-473, 70 (1994b) 475-484], with the added assumption that one process (period correction) depends on conscious awareness of a tempo change whereas the other (phase correction) does not. This assumption received support in Experiment 2, where a synchronization-continuation tapping task was used in combination with perceptual judgments to probe into the process of period correction following step changes. The results led to the conclusion that rapid adaptation of the tapping period to a small, undetected tempo change is in fact due to rapid internal phase correction, whereas slow adaptation of the relative phase of the taps is due to slow internal period correction.

Cues to the perception of Taiwanese tones.

A labeling test with synthetic speech stimuli was carried out to determine to what extent the two dimensions of fundamental frequency (F0), height and movement, and syllable duration provide cues to tonal distinctions in Taiwanese. The data show that the high level vs. mid level tones and the high falling vs. mid falling tones can be reliably distinguished by F0 height alone, whereas the distinction between tones with dissimilar contours, such as the high falling and low rising tones, is predominantly cued by F0 movement. However, the other dimension of F0 may collaborate with the dominant one in cueing a tonal contrast, depending on the extent to which the two tones differ along that dimension. Syllable duration has a small additional effect on the perception of the distinction between falling and nonfalling tones. These results are consistent with previous findings in tone languages other than Taiwanese in that they suggest that tones are mainly cued by F0. While the primacy of F0 dimensions as cues to tonal contrasts depends on the contrast to be distinguished, the present findings show that tones which nominally differ only in register (e.g., high falling vs. mid falling) may exhibit perceptually relevant contour differences, and vice versa.

The sound of two hands clapping: an exploratory study.

Clapping is a little-studied human activity that may be viewed either as a form of communicative group behavior (applause) or as an individual sound-generating activity involving two "articulators"--the hands. The latter aspect was explored in this pilot study by means of acoustical analyses and perceptual experiments. Principal components analysis of 20 subjects' average clap spectra yielded several dimensions of interindividual variation that were related to observed hand configuration. This relationship emerged even more clearly in a similar analysis of a single clapper's deliberately varied productions. In perception experiments, subjects proved sensitive to spectral properties of claps: For a single clapper, at least, listeners were able to judge hand configuration with good accuracy. Besides providing some general information on individual variations in clapping, the present results support the general hypothesis that sound emanating from a natural source informs listeners about the changing states of the source mechanism.

Spectral envelope and context effects in the tritone paradox.

In previous studies of the 'tritone paradox' Deutsch has suggested that, when listeners are presented with pairs of octave-complex tones that are equal in average log frequency but differ in chroma by 6 semitones (a tritone), they perceive the direction of the chroma difference according to an individual pitch-class template. However, it has also been found that the perceived direction changes for many listeners when the spectral envelope of the tones is shifted along the frequency axis. Reanalysis of these data indicates a strong tendency to perceive the pitch class corresponding to the frequency on which the spectral envelope is centered as subjectively lowest. In experiment 1 this spectral-envelope effect was replicated with tone pairs presented in isolation, at the rate of one a day, which rules out artifacts of test format. In experiment 2, involving another context-free format, envelope center frequency was varied over a wide range and it was shown that some individuals are totally envelope dependent, whereas others rely more on pitch class, and yet others show mixed patterns. Experiment 3 demonstrated that listeners' judgments of tritone pairs can be swayed easily by preceding context. Finally, experiment 4 showed that strong envelope effects are also obtained with Deutsch's own tritone test (issued on CD). The subjective relative pitch height of octave-complex tones thus depends on several competing factors, only one of which is pitch class.