Transitional Probabilities Are Prioritized over Stimulus/Pattern Probabilities in Auditory Deviance Detection: Memory Basis for Predictive Sound Processing.

UNLABELLED: Representations encoding the probabilities of auditory events do not directly support predictive processing. In contrast, information about the probability with which a given sound follows another (transitional probability) allows predictions of upcoming sounds. We tested whether behavioral and cortical auditory deviance detection (the latter indexed by the mismatch negativity event-related potential) relies on probabilities of sound patterns or on transitional probabilities. We presented healthy adult volunteers with three types of rare tone-triplets among frequent standard triplets of high-low-high (H-L-H) or L-H-L pitch structure: proximity deviant (H-H-H/L-L-L), reversal deviant (L-H-L/H-L-H), and first-tone deviant (L-L-H/H-H-L). If deviance detection was based on pattern probability, reversal and first-tone deviants should be detected with similar latency because both differ from the standard at the first pattern position. If deviance detection was based on transitional probabilities, then reversal deviants should be the most difficult to detect because, unlike the other two deviants, they contain no low-probability pitch transitions. The data clearly showed that both behavioral and cortical auditory deviance detection uses transitional probabilities. Thus, the memory traces underlying cortical deviance detection may provide a link between stimulus probability-based change/novelty detectors operating at lower levels of the auditory system and higher auditory cognitive functions that involve predictive processing. SIGNIFICANCE STATEMENT: Our research presents the first definite evidence for the auditory system prioritizing transitional probabilities over probabilities of individual sensory events. Forming representations for transitional probabilities paves the way for predictions of upcoming sounds. Several recent theories suggest that predictive processing provides the general basis of human perception, including important auditory functions, such as auditory scene analysis. Our results demonstrate that the memory traces underlying cortical deviance detection form a link between stimulus probability-based change/novelty detectors operating at lower levels of the auditory system and higher auditory cognitive functions that involve predictive processing.

The effects of visual material and temporal synchrony on the processing of letters and speech sounds.

Associating letters with speech sounds is essential for reading skill acquisition. In the current study, we aimed at determining the effects of different types of visual material and temporal synchrony on the integration of letters and speech sounds. To this end, we recorded the mismatch negativity (MMN), an index of automatic change detection in the brain, from literate adults. Subjects were presented with auditory consonant-vowel syllable stimuli together with visual stimuli, which were either written syllables or scrambled pictures of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half 200 ms before the auditory stimuli. The auditory stimuli were consonant, vowel or vowel length changes, or changes in syllable frequency or intensity presented by using the multi-feature paradigm. Changes in the auditory stimuli elicited MMNs in all conditions. MMN amplitudes for the consonant and frequency changes were generally larger for the sounds presented with written syllables than with scrambled syllables. Time delay diminished the MMN amplitude for all deviants. The results suggest that speech sound processing is modulated when the sounds are presented with letters versus non-linguistic visual stimuli, and further, that the integration of letters and speech sounds seems to be dependent on precise temporal alignment. Moreover, the results indicate that with our paradigm, a variety of parameters relevant and irrelevant for reading can be tested within one experiment.

Parameter-specific modulation of the mismatch negativity to duration decrement and increment: evidence for asymmetric processes.

OBJECTIVE: The mismatch negativity (MMN) component of event-related potentials (ERPs) reflects a change-detection process in the brain. The present study investigated whether stimulus parameters (sound type and duration) exert a differential influence on the MMN for a duration decrement and increment of an equal magnitude. Some asymmetries were reported in the previous studies; yet no systematical study has been conducted. METHODS: ERPs were recorded from 16 healthy adults presented with repetitive standard sounds interspersed with duration changes (deviant sounds). In separate sequences, stimuli were vowels, music chord, sinusoid, or band-pass filtered white noise. The stimulus durations (standard/deviant) were either 200/120 ms or 400/240 ms for decrements, and vice versa for increments. RESULTS: The MMN for the increments was abolished in the 400/240 ms condition, whereas the MMN for decrements was significant irrespective of the sound duration. The amplitude of the increment MMN paralleled with the spectral complexity of the stimulus sound, whereas that of the decrement MMN was larger for natural sounds than artificial sounds. CONCLUSIONS: The observed interactions demonstrated asymmetries in the MMN for duration increment and decrement. SIGNIFICANCE: The present findings suggest that the effects of stimulus parameters should be taken into account when comparing different studies, especially where clinical populations are involved, with one another.

Preattentive detection of nonsalient contingencies between auditory features.

Extraction of contingencies between features of successive auditory stimuli was studied by exploiting the mismatch negativity component of the event-related potential. According to the rules hidden in the stimulus sequences, one stimulus feature (duration) predicted another feature (pitch) of the next stimulus. Occasional deviant stimuli, violating the rules, elicited a mismatch negativity although the participants were ignoring the stimuli. Mismatch negativity was also elicited when the participants tried to detect the deviant stimuli. Their detection performance, however, was poor and on the basis of subsequent interviews, they were not consciously aware of the rules. The results suggest that contingencies across-features of successive stimuli are extracted already at the early preattentive level in the auditory sensory memory.

Measurement of extensive auditory discrimination profiles using the mismatch negativity (MMN) of the auditory event-related potential (ERP).

OBJECTIVE: Mismatch negativity (MMN), a change-specific component of the auditory event-related potential (ERP), is sensitive to deficits in central auditory processing associated with many clinical conditions. The aim of this study was to obtain a comprehensive multi-dimensional profile of central auditory processing by extending the recently developed fast multi-feature MMN paradigm [Näätänen R, Pakarinen S, Rinne T, Takegata R. The mismatch negativity (MMN): towards the optimal paradigm. Clin Neurophysiol 2004;115:140-144]. METHODS: MMN responses to changes in sound duration, frequency, intensity, and perceived sound-source location at six different magnitudes of deviation were recorded from healthy young adults by using the multi-feature MMN paradigm. In addition, behavioural discrimination accuracy and speed were measured to examine the relationship between MMN and behavioural performance. RESULTS: All the 24 sound changes elicited significant MMNs. MMN amplitude increased and latency decreased with increasing magnitude of sound change. Furthermore, the MMN amplitude and latency predicted the subjects' accuracy and speed in detecting these deviations. CONCLUSIONS: This new paradigm provides an extensive auditory discrimination profile for several auditory attributes at different deviation magnitudes in a minimal recording time. SIGNIFICANCE: The auditory discrimination profiles can offer a comprehensive view of the development, plasticity, and deficits of central auditory processing.

Event-related brain potentials reveal multiple stages in the perceptual organization of sound.

Auditory stream segregation has been suggested to include two distinct processing stages: (1) forming representations for alternative organizations of the acoustic input and (2) choosing one organization for perception after weighing the evidence that supports the different alternatives. The current study tested the possibility that auditory event-related potentials (ERP) could be used to index both stages of the stream-segregation process. Sequences of tones that could be perceived either as a single coherent auditory stream (integrated organization) or as two separate streams of sounds (segregated organization) were presented to subjects. The stimulus configuration encouraged perception to fluctuate between these alternative organizations. Subjects were instructed to continuously indicate whether they perceived one or the other organization of the tone sequence. Occasionally, a tone was omitted from the otherwise regular sequence. This deviance was expected to be processed differently depending on the perceptual organization of the sequence at the time of the omission. We found an early ERP response to omission, which was fully determined by parameters of the stimulation and was not sensitive to the perceived sound organization. In contrast, modulation of two ERP components elicited by the regular tone patterns as well as later responses elicited by deviants correlated with the perceived sound organization. These results suggest that sound organization goes through at least two distinct stages, the first being fully stimulus driven, whereas the second is partly under top-down control.

Preattentive representation of feature conjunctions for concurrent spatially distributed auditory objects.

The role of attention in conjoining features of an object has been a topic of much debate. Studies using the mismatch negativity (MMN), an index of detecting acoustic deviance, suggested that the conjunctions of auditory features are preattentively represented in the brain. These studies, however, used sequentially presented sounds and thus are not directly comparable with visual studies of feature integration. Therefore, the current study presented an array of spatially distributed sounds to determine whether the auditory features of concurrent sounds are correctly conjoined without focal attention directed to the sounds. Two types of sounds differing from each other in timbre and pitch were repeatedly presented together while subjects were engaged in a visual n-back working-memory task and ignored the sounds. Occasional reversals of the frequent pitch-timbre combinations elicited MMNs of a very similar amplitude and latency irrespective of the task load. This result suggested preattentive integration of auditory features. However, performance in a subsequent target-search task with the same stimuli indicated the occurrence of illusory conjunctions. The discrepancy between the results obtained with and without focal attention suggests that illusory conjunctions may occur during voluntary access to the preattentively encoded object representations.

Effects of temporal grouping on the memory representation of inter-tone relationships.

The length of silence between successive sounds is a dominant cue for temporal grouping of sounds. The present study tested whether the sensory memory representation of inter-tone relationships is dependent on the grouping of tones within a single stream of sound. Subjects were presented with sequences of two alternating tones that differed from each other in frequency. Perception of a sequence made up of tone-pairs was promoted by alternating a short and a long inter-tone interval. Occasional tone repetitions fell either within one tone-pair or across two pairs. We found that detecting tone repetitions was slower for across- than within-pair repetitions (Experiment 1). Also the amplitude of the mismatch negativity (MMN) event-related potential was lower for across-pair repetitions compared with that measured in the control isochronous sequences (Experiment 2). This attenuation of the MMN-amplitude could not be explained by the inter-tone interval differences that existed between the paired and the isochronous conditions (Experiment 3). These results demonstrate that temporal grouping affects the sensory memory representation of inter-tone relationships within a single sound stream.

Spectral and temporal stimulus characteristics in the processing of abstract auditory features.

The processing of abstract stimulus features in the human brain was studied by presenting the subjects with frequent standard tone pairs and infrequent deviant tone pairs. Both pairs varied randomly over a wide frequency and/or intensity range, there being no physically constant standard stimulus. The common feature of the standard pairs was the direction of change within the pair, e.g. the second tone was louder in intensity and/or higher in frequency than the first tone. Deviant pairs, having opposite feature-change direction, elicited the mismatch-negativity (MMN) event-related potential component. MMN was similar to deviations in the direction of frequency and intensity changes and showed no additivity for simultaneous changes in both feature directions. Moreover, MMN was elicited even when the within-pair interval exceeded the 200 ms limit of auditory temporal integration. Results demonstrate that extraction of abstract features is not limited to frequency-based rules, nor is it dependent on temporal integration mechanisms. The lack of MMN additivity between violations of multiple abstract rules suggests that the processing of higher-order invariances differs from that of simple physical features.

Hemispheric processing of duration changes in speech and non-speech sounds.

Sound duration conveys phonemic information in some languages. The present study, using magnetoencephalography (MEG), examined whether the hemispheric activation associated with the processing of duration is different between speech and non-speech sounds in subjects whose native language uses duration as a phonemic cue. The magnetic mismatch negativity (MMNm) response was recorded for equal-duration decrements in vowel, sinusoidal, and spectrally rich complex sounds. Although the MMNm responses to duration changes were predominant in the right hemisphere, the distribution of this response for the vowel stimuli was significantly displaced leftward compared with that for the other two types of stimuli. The results suggest that the hemispheric distribution of the MMNm response to duration change depends on the linguistic relevance of the change.

The mismatch negativity (MMN): towards the optimal paradigm.

OBJECTIVE: Recent studies have shown that the mismatch negativity (MMN), a change-specific component of the event-related potential (ERP), for particular auditory features is degraded in different clinical populations. This suggests that the MMN could, in principle, reflect the whole profile and extent of the central auditory deficit. In the present article, we tested a new MMN paradigm allowing one to obtain MMNs for several auditory attributes in a short time. METHODS: MMN responses to changes in frequency, intensity, duration, location, and to a silent gap occasionally inserted in the middle of a tone were compared between the traditional 'oddball' paradigm (a single type of auditory change in each sequence) and the new paradigm (two versions) in which all the 5 types of changes appeared within the same sequence. RESULTS: The MMNs obtained in the new paradigm were equal in amplitude to those in the traditional MMN paradigm. CONCLUSIONS: We propose a new paradigm that can provide 5 different MMNs in the same time in which usually only one MMN is obtained. The new paradigm enables one to objectively determine the profile of different auditory discrimination abilities within a very short recording time.

Audiovisual attention boosts letter-speech sound integration.

We studied attention effects on the integration of written and spoken syllables in fluent adult readers by using event-related brain potentials. Auditory consonant-vowel syllables, including consonant and frequency changes, were presented in synchrony with written syllables or their scrambled images. Participants responded to longer-duration auditory targets (auditory attention), longer-duration visual targets (visual attention), longer-duration auditory and visual targets (audiovisual attention), or counted backwards mentally. We found larger negative responses for spoken consonant changes when they were accompanied by written syllables than when they were accompanied by scrambled text. This effect occurred at an early latency (∼ 140 ms) during audiovisual attention and later (∼ 200 ms) during visual attention. Thus, audiovisual attention boosts the integration of speech sounds and letters.

Sound energy and the magnitude of change: effects on mismatch negativity.

The mismatch negativity (MMN) of event-related potential generally increases in amplitude, as a function of magnitude of change. This study examined whether this relation holds true for intensity decrement, in which the stimulus energy that conveys the change falls in inverse proportion to magnitude of change. The MMN was recorded from healthy young adults for intensity decrements of 10, 20, 30, 40, and 50 dB. As the change increased, the MMN amplitude also increased first and thereafter diminished; thus, an inverted U-shaped relation was found between the MMN amplitude and the magnitude of change. These results, therefore, suggest a possible interplay between the energy of deviant stimulus and magnitude of change in the MMN elicitation.

Neural process underlying gap detection for spectrally rich and asymmetrical markers.

Processing of silent gaps of the order of milliseconds is crucial in speech perception. To investigate such process, we compared gap detection for spectrally symmetrical, slightly or widely asymmetrical markers, using the mismatch negativity (MMN), an index of preattentive change detection in the brain. The slightly asymmetrical markers declined the MMN amplitude alone, but the widely asymmetrical markers affected both the MMN amplitude and latency, suggesting that the influence of spectrally asymmetrical markers on gap detection is not uniform across different magnitude of asymmetries. In contrast to the prevailing view, the MMN obtained indicated that the effects of marker asymmetry took place as early as at preattentive stage.

Pre-attentive representation of sound duration in the human brain.

Studies using a brain index for pre-attentive change detection, the mismatch negativity (MMN), suggested distinct neuronal populations for signaling changes in sound duration and frequency. However, these studies used only durations within the temporal window of loudness summation (ca. 200 ms) in which any duration change is accompanied by a loudness change. Hence, the present study employed stimulus durations both beyond and within this temporal window in order to examine the genuine duration representation in the brain. Magnetic mismatch responses (MMNm) for duration and frequency changes were compared with each other. The equivalent current dipole (ECD) of the duration MMNm was located in the auditory cortex slightly posterior to that for the frequency MMNm irrespective of stimulus duration. The results suggested separate memory representations for sound duration and frequency in the human brain.

Automatic time perception in the human brain for intervals ranging from milliseconds to seconds.

Time perception in everyday life deals with various intervals. Here we investigated whether an automatic duration-discrimination mechanism in audition operates even for intervals of an order of seconds, by using the mismatch negativity (MMN), an index of automatic change detection in audition. In Experiment 1, occasional decrements of the duration of a repetitive "standard" tone elicited an MMN in subjects ignoring auditory stimulation, even with the standard-stimulus durations over a second. Nevertheless, the MMN amplitude was significantly diminished with standard-stimulus durations of 800 ms and above, despite the fact that a constant deviant versus standard duration ratio was used. Complementary experiments varying the interstimulus interval (Experiment 2) and the magnitude of duration change (Experiment 3) yielded corroborating results. The present results suggest that automatic duration discrimination in audition operates even for durations of the order of seconds; yet its optimum time scale might be of the order of milliseconds.

A paradigm to measure mismatch negativity responses to phonetic and acoustic changes in parallel.

The mismatch negativity (MMN) of the event-related potential has been purported to be an objective index of central auditory processing. The present study tested a new paradigm to measure the MMN responses to phonological changes in parallel with those to simple acoustic changes. Stimulus sequences consisted of repetitive consonant-vowel syllables interspersed with infrequent phonetic changes (in place of articulation or voicing) and repetitive sinusoidal tones with occasional acoustic changes (in frequency or duration). The speech and tone stimuli were delivered to the opposite ears (right and left, respectively) at a stimulus onset asynchrony (SOA) of 300 ms. The MMNs elicited in this new paradigm were compared with those measured in a conventional paradigm, in which the speech and tone stimuli were presented in separate sequences at an identical speech-speech or tone-tone SOA (600 ms) of the new paradigm. The MMNs elicited in the two paradigms had a similar morphology and topography, although the MMNs measured with the new paradigm were slightly smaller for 3 out of 4 types of deviants. The results suggest that the new paradigm enables the measurement of reliable MMNs to phonetic changes in parallel with those to acoustic changes.