The role of carrier spectral composition in the perception of musical pitch.

Temporal envelope fluctuations of natural sounds convey critical information to speech and music processing. In particular, musical pitch perception is assumed to be primarily underlined by temporal envelope encoding. While increasing evidence demonstrates the importance of carrier fine structure to complex pitch perception, how carrier spectral information affects musical pitch perception is less clear. Here, transposed tones designed to convey identical envelope information across different carriers were used to assess the effects of carrier spectral composition to pitch discrimination and musical-interval and melody identifications. Results showed that pitch discrimination thresholds became lower (better) with increasing carrier frequencies from 1k to 10k Hz, with performance comparable to that of pure sinusoids. Musical interval and melody defined by the periodicity of sine- or harmonic complex envelopes across carriers were identified with greater than 85% accuracy even on a 10k-Hz carrier. Moreover, enhanced interval and melody identification performance was observed with increasing carrier frequency up to 6k Hz. Findings suggest a perceptual enhancement of temporal envelope information with increasing carrier spectral region in musical pitch processing, at least for frequencies up to 6k Hz. For carriers in the extended high-frequency region (8-20k Hz), the use of temporal envelope information to music pitch processing may vary depending on task requirement. Collectively, these results implicate the fidelity of temporal envelope information to musical pitch perception is more pronounced than previously considered, with ecological implications.

Crossmodal Associations with Olfactory, Auditory, and Tactile Stimuli in Children and Adults.

People associate information with different senses but the mechanism by which this happens is unclear. Such associations are thought to arise from innate structural associations in the brain, statistical associations in the environment, via shared affective content, or through language. A developmental perspective on crossmodal associations can help determine which explanations are more likely for specific associations. Certain associations with pitch (e.g., pitch-height) have been observed early in infancy, but others may only occur late into childhood (e.g., pitch-size). In contrast, tactile-chroma associations have been observed in children, but not adults. One modality that has received little attention developmentally is olfaction. In the present investigation, we explored crossmodal associations from sound, tactile stimuli, and odor to a range of stimuli by testing a broad range of participants. Across the three modalities, we found little evidence for crossmodal associations in young children. This suggests an account based on innate structures is unlikely. Instead, the number and strength of associations increased over the lifespan. This suggests that experience plays a crucial role in crossmodal associations from sound, touch, and smell to other senses.

Space-pitch associations differ in their susceptibility to language.

To what extent are links between musical pitch and space universal, and to what extent are they shaped by language? There is contradictory evidence in support of both universality and linguistic relativity presently, leaving the question open. To address this, speakers of Dutch who talk about pitch in terms of spatial height and speakers of Turkish who use a thickness metaphor were tested in simple nonlinguistic space-pitch association tasks. Both groups showed evidence of a thickness-pitch association, but differed significantly in their height-pitch associations, suggesting the latter may be more susceptible to language. When participants had to match pitches to spatial stimuli where height and thickness were opposed (i.e., a thick line high in space vs. a thin line low in space), Dutch and Turkish differed in their relative preferences. Whereas Turkish participants predominantly opted for a thickness-pitch interpretation-even if this meant a reversal of height-pitch mappings-Dutch participants favored a height-pitch interpretation more often. These findings provide new evidence that speakers of different languages vary in their space-pitch associations, while at the same time showing such associations are not equally susceptible to linguistic influences. Some space-pitch (i.e., height-pitch) associations are more malleable than others (i.e., thickness-pitch).

Are lexical tones musical? Native language's influence on neural response to pitch in different domains.

Language experience shapes musical and speech pitch processing. We investigated whether speaking a lexical tone language natively modulates neural processing of pitch in language and music as well as their correlation. We tested tone language (Mandarin Chinese), and non-tone language (Dutch) listeners in a passive oddball paradigm measuring mismatch negativity (MMN) for (i) Chinese lexical tones and (ii) three-note musical melodies with similar pitch contours. For lexical tones, Chinese listeners showed a later MMN peak than the non-tone language listeners, whereas for MMN amplitude there were no significant differences between groups. Dutch participants also showed a late discriminative negativity (LDN). In the music condition two MMNs, corresponding to the two notes that differed between the standard and the deviant were found for both groups, and an LDN were found for both the Dutch and the Chinese listeners. The music MMNs were significantly right lateralized. Importantly, significant correlations were found between the lexical tone and the music MMNs for the Dutch but not the Chinese participants. The results suggest that speaking a tone language natively does not necessarily enhance neural responses to pitch either in language or in music, but that it does change the nature of neural pitch processing: non-tone language speakers appear to perceive lexical tones as musical, whereas for tone language speakers, lexical tones and music may activate different neural networks. Neural resources seem to be assigned differently for the lexical tones and for musical melodies, presumably depending on the presence or absence of long-term phonological memory traces.

Pitch Perception in the First Year of Life, a Comparison of Lexical Tones and Musical Pitch.

Pitch variation is pervasive in speech, regardless of the language to which infants are exposed. Lexical tone is influenced by general sensitivity to pitch. We examined whether the development in lexical tone perception may develop in parallel with perception of pitch in other cognitive domains namely music. Using a visual fixation paradigm, 100 and one 4- and 12-month-old Dutch infants were tested on their discrimination of Chinese rising and dipping lexical tones as well as comparable three-note musical pitch contours. The 4-month-old infants failed to show a discrimination effect in either condition, whereas the 12-month-old infants succeeded in both conditions. These results suggest that lexical tone perception may reflect and relate to general pitch perception abilities, which may serve as a basis for developing more complex language and musical skills.

Attentional modulation and domain-specificity underlying the neural organization of auditory categorical perception.

Categorical perception (CP) is highly evident in audition when listeners' perception of speech sounds abruptly shifts identity despite equidistant changes in stimulus acoustics. While CP is an inherent property of speech perception, how (if) it is expressed in other auditory modalities (e.g., music) is less clear. Moreover, prior neuroimaging studies have been equivocal on whether attentional engagement is necessary for the brain to categorically organize sound. To address these questions, we recorded neuroelectric brain responses [event-related potentials (ERPs)] from listeners as they rapidly categorized sounds along a speech and music continuum (active task) or during passive listening. Behaviorally, listeners' achieved sharper psychometric functions and faster identification for speech than musical stimuli, which was perceived in a continuous mode. Behavioral results coincided with stronger ERP differentiation between prototypical and ambiguous tokens (i.e., categorical processing) for speech but not for music. Neural correlates of CP were only observed when listeners actively attended to the auditory signal. These findings were corroborated by brain-behavior associations; changes in neural activity predicted more successful CP (psychometric slopes) for active but not passively evoked ERPs. Our results demonstrate auditory categorization is influenced by attention (active > passive) and is stronger for more familiar/overlearned stimulus domains (speech > music). In contrast to previous studies examining highly trained listeners (i.e., musicians), we infer that (i) CP skills are largely domain-specific and do not generalize to stimuli for which a listener has no immediate experience and (ii) categorical neural processing requires active engagement with the auditory stimulus.

Trombones Elicit Bitter More Strongly Than Do Clarinets: a Partial Replication of Three Studies of Crisinel and Spence.

We present a partial replication of the crossmodal pitch/taste correspondence of Crisinel and Spence. Male college students (n=46) were asked to judge the pitch (F1-C4 on trombone; F3-C6 on clarinet) that best corresponded with each of four tastants (unsweetened coffee, unsweetened chocolate, salt, and sugar). With trombone there was a significant effect of tastant [F(3,135)=7.574, p<0.001, η2=0.144] with unsweetened chocolate being associated with the lowest pitch and sugar with the highest. With clarinet we did not find a significant effect [F(3,135)=2.468, p=0.065, η2=0.052]. The average across instruments was significant [F(3,135)=4.269, p=0.006, η2=0.087]. When looking at the effect of taster status, we found a significant correlation [r(44)=0.389, p=0.007] with supertasters associating the bitterness of a PTC strip with higher pitches than did nontasters - this is in contrast with Crisinel and Spence's finding of no correlation with taster status. In light of the 'replication crisis' in psychology as found by the Open Science Collaboration, it is noteworthy that this crossmodal pitch/taste correspondence, at least for trombone, was replicated in a different lab.

Subgroup differences in the lexical tone mismatch negativity (MMN) among Mandarin speakers with congenital amusia.

The association/dissociation of pitch processing between music and language is a long lasting debate. We examined this music-language relationship by investigating to what extent pitch deficits in these two domains were dissociable. We focused on a special neurodevelopmental pitch disorder - congenital amusia, which primarily affects musical pitch processing. Recent research has also revealed lexical tone deficits in speech among amusics. Approximately one-third of Mandarin amusics exhibits behavioural difficulties in lexical tone perception, which is known as tone agnosia. Using mismatch negativities (MMNs), our current work probed lexical tone encoding at the pre-attentive level among the Mandarin amusics with (tone agnosics) and without (pure amusics) behavioural lexical tone deficits compared with age- and IQ-matched controls. Relative to the controls and the pure amusics, the tone agnosics exhibited reduced MMNs specifically in response to lexical tone changes. Their tone-consonant MMNs were intact and similar to those of the other two groups. Moreover, the tone MMN reduction over the left hemisphere was tightly linked to behavioural insensitivity to lexical tone changes. The current study thus provides the first psychophysiological evidence of subgroup differences in lexical tone processing among Mandarin amusics and links amusics' behavioural tone deficits to impaired pre-attentive tone processing. Despite the overall music pitch deficits, the subgroup differences in lexical tone processing in Mandarin-speaking amusics suggest dissociation of pitch deficits between music and speech.

Spatial Congruity Effects Reveal Metaphorical Thinking, not Polarity Correspondence.

Spatial congruity effects have often been interpreted as evidence for metaphorical thinking, but an alternative account based on polarity correspondence (a.k.a. markedness) has challenged this view. Here we compared metaphor- and polarity-correspondence-based explanations for spatial congruity effects, using musical pitch as a testbed. In one experiment, English speakers classified high- and low-frequency pitches as "high" and "low," or as "front" and "back," to determine whether space-pitch congruity effects could be elicited by any marked spatial continuum. Although both pairs of terms describe bipolar spatial continuums, we found congruity effects only for high/low judgments, indicating that markedness is not sufficient to produce space-pitch congruity effects. A second experiment confirmed that there were no space-pitch congruity effects for another pair of terms that have clear markedness (big/small), but which do not denote spatial height. By contrast, this experiment showed congruity effects for words that cued an appropriate vertical spatial schema (tall/short), even though these words are not used conventionally in English to describe pitches, ruling out explanations for the observed pattern of results based on verbal polysemy. Together, results suggest that space-pitch congruity effects reveal metaphorical uses of spatial schemas, not polarity correspondence effects.

Absence of modulatory action on haptic height perception with musical pitch.

Although acoustic frequency is not a spatial property of physical objects, in common language, pitch, i.e., the psychological correlated of frequency, is often labeled spatially (i.e., "high in pitch" or "low in pitch"). Pitch-height is known to modulate (and interact with) the response of participants when they are asked to judge spatial properties of non-auditory stimuli (e.g., visual) in a variety of behavioral tasks. In the current study we investigated whether the modulatory action of pitch-height extended to the haptic estimation of height of a virtual step. We implemented a HW/SW setup which is able to render virtual 3D objects (stair-steps) haptically through a PHANTOM device, and to provide real-time continuous auditory feedback depending on the user interaction with the object. The haptic exploration was associated with a sinusoidal tone whose pitch varied as a function of the interaction point's height within (i) a narrower and (ii) a wider pitch range, or (iii) a random pitch variation acting as a control audio condition. Explorations were also performed with no sound (haptic only). Participants were instructed to explore the virtual step freely, and to communicate height estimation by opening their thumb and index finger to mimic the step riser height, or verbally by reporting the height in centimeters of the step riser. We analyzed the role of musical expertise by dividing participants into non-musicians and musicians. Results showed no effects of musical pitch on high-realistic haptic feedback. Overall there is no difference between the two groups in the proposed multimodal conditions. Additionally, we observed a different haptic response distribution between musicians and non-musicians when estimations of the auditory conditions are matched with estimations in the no sound condition.

Prelinguistic infants are sensitive to space-pitch associations found across cultures.

People often talk about musical pitch using spatial metaphors. In English, for instance, pitches can be "high" or "low" (i.e., height-pitch association), whereas in other languages, pitches are described as "thin" or "thick" (i.e., thickness-pitch association). According to results from psychophysical studies, metaphors in language can shape people's nonlinguistic space-pitch representations. But does language establish mappings between space and pitch in the first place, or does it only modify preexisting associations? To find out, we tested 4-month-old Dutch infants' sensitivity to height-pitch and thickness-pitch mappings using a preferential-looking paradigm. The infants looked significantly longer at cross-modally congruent stimuli for both space-pitch mappings, which indicates that infants are sensitive to these associations before language acquisition. The early presence of space-pitch mappings means that these associations do not originate from language. Instead, language builds on preexisting mappings, changing them gradually via competitive associative learning. Space-pitch mappings that are language-specific in adults develop from mappings that may be universal in infants.

The role of the auditory brainstem in processing musically relevant pitch.

Neuroimaging work has shed light on the cerebral architecture involved in processing the melodic and harmonic aspects of music. Here, recent evidence is reviewed illustrating that subcortical auditory structures contribute to the early formation and processing of musically relevant pitch. Electrophysiological recordings from the human brainstem and population responses from the auditory nerve reveal that nascent features of tonal music (e.g., consonance/dissonance, pitch salience, harmonic sonority) are evident at early, subcortical levels of the auditory pathway. The salience and harmonicity of brainstem activity is strongly correlated with listeners' perceptual preferences and perceived consonance for the tonal relationships of music. Moreover, the hierarchical ordering of pitch intervals/chords described by the Western music practice and their perceptual consonance is well-predicted by the salience with which pitch combinations are encoded in subcortical auditory structures. While the neural correlates of consonance can be tuned and exaggerated with musical training, they persist even in the absence of musicianship or long-term enculturation. As such, it is posited that the structural foundations of musical pitch might result from innate processing performed by the central auditory system. A neurobiological predisposition for consonant, pleasant sounding pitch relationships may be one reason why these pitch combinations have been favored by composers and listeners for centuries. It is suggested that important perceptual dimensions of music emerge well before the auditory signal reaches cerebral cortex and prior to attentional engagement. While cortical mechanisms are no doubt critical to the perception, production, and enjoyment of music, the contribution of subcortical structures implicates a more integrated, hierarchically organized network underlying music processing within the brain.

The thickness of musical pitch: psychophysical evidence for linguistic relativity.

Do people who speak different languages think differently, even when they are not using language? To find out, we used nonlinguistic psychophysical tasks to compare mental representations of musical pitch in native speakers of Dutch and Farsi. Dutch speakers describe pitches as high (hoog) or low (laag), whereas Farsi speakers describe pitches as thin (nazok) or thick (koloft). Differences in language were reflected in differences in performance on two pitch-reproduction tasks, even though the tasks used simple, nonlinguistic stimuli and responses. To test whether experience using language influences mental representations of pitch, we trained native Dutch speakers to describe pitch in terms of thickness, as Farsi speakers do. After the training, Dutch speakers' performance on a nonlinguistic psychophysical task resembled the performance of native Farsi speakers. People who use different linguistic space-pitch metaphors also think about pitch differently. Language can play a causal role in shaping nonlinguistic representations of musical pitch.

Evidence for pitch chroma mapping in human auditory cortex.

Some areas in auditory cortex respond preferentially to sounds that elicit pitch, such as musical sounds or voiced speech. This study used human electroencephalography (EEG) with an adaptation paradigm to investigate how pitch is represented within these areas and, in particular, whether the representation reflects the physical or perceptual dimensions of pitch. Physically, pitch corresponds to a single monotonic dimension: the repetition rate of the stimulus waveform. Perceptually, however, pitch has to be described with 2 dimensions, a monotonic, "pitch height," and a cyclical, "pitch chroma," dimension, to account for the similarity of the cycle of notes (c, d, e, etc.) across different octaves. The EEG adaptation effect mirrored the cyclicality of the pitch chroma dimension, suggesting that auditory cortex contains a representation of pitch chroma. Source analysis indicated that the centroid of this pitch chroma representation lies somewhat anterior and lateral to primary auditory cortex.

Perceptual pitch deficits coexist with pitch production difficulties in music but not Mandarin speech.

Congenital amusia is a musical disorder that mainly affects pitch perception. Among Mandarin speakers, some amusics also have difficulties in processing lexical tones (tone agnosics). To examine to what extent these perceptual deficits may be related to pitch production impairments in music and Mandarin speech, eight amusics, eight tone agnosics, and 12 age- and IQ-matched normal native Mandarin speakers were asked to imitate music note sequences and Mandarin words of comparable lengths. The results indicated that both the amusics and tone agnosics underperformed the controls on musical pitch production. However, tone agnosics performed no worse than the amusics, suggesting that lexical tone perception deficits may not aggravate musical pitch production difficulties. Moreover, these three groups were all able to imitate lexical tones with perfect intelligibility. Taken together, the current study shows that perceptual musical pitch and lexical tone deficits might coexist with musical pitch production difficulties. But at the same time these perceptual pitch deficits might not affect lexical tone production or the intelligibility of the speech words that were produced. The perception-production relationship for pitch among individuals with perceptual pitch deficits may be, therefore, domain-dependent.