The paradoxical role of emotional intensity in the perception of vocal affect.

Vocalizations including laughter, cries, moans, or screams constitute a potent source of information about the affective states of others. It is typically conjectured that the higher the intensity of the expressed emotion, the better the classification of affective information. However, attempts to map the relation between affective intensity and inferred meaning are controversial. Based on a newly developed stimulus database of carefully validated non-speech expressions ranging across the entire intensity spectrum from low to peak, we show that the intuition is false. Based on three experiments (N = 90), we demonstrate that intensity in fact has a paradoxical role. Participants were asked to rate and classify the authenticity, intensity and emotion, as well as valence and arousal of the wide range of vocalizations. Listeners are clearly able to infer expressed intensity and arousal; in contrast, and surprisingly, emotion category and valence have a perceptual sweet spot: moderate and strong emotions are clearly categorized, but peak emotions are maximally ambiguous. This finding, which converges with related observations from visual experiments, raises interesting theoretical challenges for the emotion communication literature.

Acoustically Driven Cortical δ Oscillations Underpin Prosodic Chunking.

Oscillation-based models of speech perception postulate a cortical computational principle by which decoding is performed within a window structure derived by a segmentation process. Segmentation of syllable-size chunks is realized by a θ oscillator. We provide evidence for an analogous role of a δ oscillator in the segmentation of phrase-sized chunks. We recorded magnetoencephalography (MEG) in humans, while participants performed a target identification task. Random-digit strings, with phrase-long chunks of two digits, were presented at chunk rates of 1.8 or 2.6 Hz, inside or outside the δ frequency band (defined here to be 0.5-2 Hz). Strong periodicities were elicited by chunk rates inside of δ in superior, middle temporal areas and speech-motor integration areas. Periodicities were diminished or absent for chunk rates outside δ, in line with behavioral performance. Our findings show that prosodic chunking of phrase-sized acoustic segments is correlated with acoustic-driven δ oscillations, expressing anatomically specific patterns of neuronal periodicities.

Corrigendum: The Lateralization of Speech-Brain Coupling Is Differentially Modulated by Intrinsic Auditory and Top-Down Mechanisms.

[This corrects the article DOI: 10.3389/fnint.2019.00028.].

The Lateralization of Speech-Brain Coupling Is Differentially Modulated by Intrinsic Auditory and Top-Down Mechanisms.

The lateralization of neuronal processing underpinning hearing, speech, language, and music is widely studied, vigorously debated, and still not understood in a satisfactory manner. One set of hypotheses focuses on the temporal structure of perceptual experience and links auditory cortex asymmetries to underlying differences in neural populations with differential temporal sensitivity (e.g., ideas advanced by Zatorre et al. (2002) and Poeppel (2003). The Asymmetric Sampling in Time theory (AST) (Poeppel, 2003), builds on cytoarchitectonic differences between auditory cortices and predicts that modulation frequencies within the range of, roughly, the syllable rate, are more accurately tracked by the right hemisphere. To date, this conjecture is reasonably well supported, since - while there is some heterogeneity in the reported findings - the predicted asymmetrical entrainment has been observed in various experimental protocols. Here, we show that under specific processing demands, the rightward dominance disappears. We propose an enriched and modified version of the asymmetric sampling hypothesis in the context of speech. Recent work (Rimmele et al., 2018b) proposes two different mechanisms to underlie the auditory tracking of the speech envelope: one derived from the intrinsic oscillatory properties of auditory regions; the other induced by top-down signals coming from other non-auditory regions of the brain. We propose that under non-speech listening conditions, the intrinsic auditory mechanism dominates and thus, in line with AST, entrainment is rightward lateralized, as is widely observed. However, (i) depending on individual brain structural/functional differences, and/or (ii) in the context of specific speech listening conditions, the relative weight of the top-down mechanism can increase. In this scenario, the typically observed auditory sampling asymmetry (and its rightward dominance) diminishes or vanishes.

A template-free approach for determining the latency of single events of auditory evoked M100.

The phase of the complex output of a narrow band Gaussian filter is taken to define the latency of the auditory evoked response M100 recorded by magnetoencephalography. It is demonstrated that this definition is consistent with the conventional peak latency. Moreover, it provides a tool for reducing the number of averages needed for a reliable estimation of the latency. Single-event latencies obtained by this procedure can be used to improve the signal quality of the conventional average by latency adjusted averaging.

Task-induced asymmetry of the auditory evoked M100 neuromagnetic field elicited by speech sounds.

The auditory evoked neuromagnetic fields elicited by synthesized speech sounds (consonant-vowel syllables) were recorded in six subjects over the left and right temporal cortices using a 37-channel SQUID-based magnetometer. The latencies and amplitudes of the peaks of the M100 evoked responses were bilaterally symmetric for passively presented stimuli. In contrast, when subjects were asked to discriminate among the same syllabic stimuli, the amplitude of the M100 increased in the left and decreased in the right temporal cortices. Single equivalent current dipole modeling of the activity elicited by all stimulus-types localized to a well-circumscribed area in supratemporal auditory cortex. The results suggest that attentional modulation affects the two supratemporal cortices in a differential manner. Task-conditioned attention to speech sounds is reflected in lateralized supratemporal cortical responses possibly concordant with hemispheric language dominance.

Latency of auditory evoked M100 as a function of tone frequency.

This study investigated post-stimulus latency of the M100 component of auditory evoked fields as a function of tone frequency. A 37-channel biomagnetometer was used to record neuromagnetic fields over the temporal lobe in response to monaurally presented tones. M100 peaks in evoked field amplitude were found for each subject. The post-stimulus latency was observed to vary parabolically with tone frequency. Latencies as short as 99 ms were found in response to mid-audio range frequencies (1000-2000 Hz), whereas lower (100-500 Hz) and higher (3000-5000 Hz) frequencies were associated with longer latencies (up to 153 ms). All fields gave M100 dipole localizations in auditory cortex. Although it was not possible to resolve spatial tonotopy, it appears that frequency information is encoded in the temporal evoked response.

Latency of evoked neuromagnetic M100 reflects perceptual and acoustic stimulus attributes.

The latency of components of the auditory evoked neuromagnetic field has been shown to reflect, or encode, stimulus attributes. In particular, the M100 component, occurring approximately 100 ms post stimulus onset has a latency that depends on stimulus pitch, spectral complexity and presentation level. This study used magnetoencephalography to record neuromagnetic fields evoked by presentation of two-tone complexes consisting of various proportions of 100 Hz and 1 kHz energy. These are perceived categorically, as evidenced by classification and reaction time measurements. It is found that the M100 latency also varies categorically, that is, characterized by two plateau regions with a sharp interface. Thus, we find that not only does the M100 latency reflect acoustic attributes of a stimulus, but also such perceptual characteristics.

Temporal encoding in auditory evoked neuromagnetic fields: stochastic resonance.

Recent investigations have demonstrated that temporal patterns of sensory neural activity detected by magnetoencephalography (MEG) reflect features of the stimulus. In this study, neuromagnetic activity was investigated using an event detection algorithm based on the correlation coefficient. The results of the technique are compared with widely used methods of analysis in two experimental conditions and are shown to identify features in the single-trial MEG response that are not apparent in the response obtained by averaging across repeated trials. As an example of the technique, the physiologic jitter in latency associated with the M100 of auditory evoked fields was reproducibly measured. Specifically, higher intensity sounds were associated with an increased reliability. The technique was also applied to the noise-enhanced evoked auditory response, producing an objective demonstration of a cortical manifestation of the phenomenon of stochastic resonance-the paradoxical enhancement in the measurement of the signal-to-noise ratio (SNR) induced by optimal addition of noise to system input.

Peri-threshold encoding of stimulus frequency and intensity in the M100 latency.

Recent work has suggested that, in addition to spatial tonotopy, pitch and timbre information may be encoded in the temporal activity of the auditory cortex. Specifically, the post-stimulus latency of the maximal cortical evoked neuromagnetic field (M100 or N1m) is a function of stimulus frequency. We investigated the additional effect of varying the stimulus intensity on the M100 response. A 37-channel biomagnetometer recorded neuromagnetic fields over the temporal lobe of healthy volunteers in response to monaurally presented tones. The frequency dependence of the M100 latency remained remarkably invariant even at low stimulus intensity. Thus, for peri-threshold stimuli, frequency information appears encoded in the temporal form of the evoked response.

Auditory evoked M100 reflects onset acoustics of speech sounds.

Magnetoencephalography (MEG) was used to investigate the response to speech sounds that differ in onset dynamics, parameterized as words that have initial stop consonants (e.g., /b/, /t/) or do not (e.g., /m/, /f/). Latency and amplitude of the M100 auditory evoked neuromagnetic field, recorded over right and left auditory cortices, varied as a function of onset: stops had shorter latencies and higher amplitudes than no-stops in both hemispheres, consistent with the hypothesis that M100 is a sensitive indicator of spectral properties of acoustic stimuli. Further, activation patterns in response to stops/no-stops differed in the two hemispheres, possibly reflecting differential perceptual processing for the acoustic-phonetic cues at the onset of spoken words.

Processing of vowels in supratemporal auditory cortex.

The auditory evoked neuromagnetic fields elicited by synthesized vowels of two different fundamental frequencies F0 were recorded in six subjects over the left and right temporal cortices using a 37-channel biomagnetometer. Single equivalent current dipole modeling of the fields elicited by all vowel types localized activity to a well-circumscribed area in supratemporal auditory cortex in both hemispheres. There were hemisphere asymmetries in the amplitude and latency of the M100 response. We also observed changes in M100 latency related to vowel type, but not to F0. There was no clear effect of vowel type or F0 on dipole localization for the M100, but a possible vowel type by latency interaction. These M100 data provide further evidence that vowels are processed independently of their pitch.

Learning transfer and neuronal plasticity in humans trained in tactile discrimination.

Adult humans were unilaterally trained in a tactile discrimination task of sequentially applied multi-finger stimuli. Magnetic source imaging (MSI) was performed before and after the training to evaluate use-dependent neuronal plasticity. All subjects showed fast improvements in performance and complete transfer of the learned task. MSI recordings revealed an unilateral decrease in current dipole strength in the somatosensory system contralateral to the trained hand. Attenuation of sensory evoked fields and a complete learning transfer indicate learning in associative and secondary cortices rather than perceptual plasticity operating on neuronal populations involved in early sensory processing. This findings are discussed with respect to an equivalent animal model and to learning specificity and generalization.

Latency of the auditory evoked neuromagnetic field components: stimulus dependence and insights toward perception.

This review will focus on investigations of the auditory evoked neuromagnetic field component, the M100, detectable in the magnetoencephalogram recorded during presentation of auditory stimuli, approximately 100 milliseconds after stimulus onset. In particular, the dependence of M100 latency on attributes of the stimulus, such as intensity, pitch and timbre will be discussed, along with evidence relating M100 latency observations to perceptual features of the stimuli. Comparison with investigation of the analogous electrical potential component, the N1, will be made. Parametric development of stimuli from pure tones through complex tones to speech elements will be made, allowing the influence of spectral pitch, virtual pitch and perceptual categorization to be delineated and suggesting implications for the role of such latency observations in the study of speech processing. The final section will deal with potential clinical applications offered by M100 latency measurements, as objective indices of normal and abnormal cortical processing.

Time-frequency MEG-MUSIC algorithm.

We propose a method that incorporates the time-frequency characteristics of neural sources into magnetoencephalographic (MEG) source estimation. The method is based on the multiple-signal-classification (MUSIC) algorithm and it calculates a time--frequency matrix in which diagonal and off-diagonal terms are the auto and crosstime--frequency distributions of multichannel MEG recordings, respectively. The method averages this time-frequency matrix over the time--frequency region of interest. The locations of neural sources are then estimated by checking the orthogonality between the noise subspace of this averaged matrix and the sensor lead field. Accordingly, the method allows us to estimate the locations of neural sources from each time--frequency component. A computer simulation was performed to test the validity of the proposed method, and the results demonstrate its effectiveness.

Noise covariance incorporated MEG-MUSIC algorithm: a method for multiple-dipole estimation tolerant of the influence of background brain activity.

This paper proposes a method of localizing multiple current dipoles from spatio-temporal biomagnetic data. The method is based on the multiple signal classification (MUSIC) algorithm and is tolerant of the influence of background brain activity. In this method, the noise covariance matrix is estimated using a portion of the data that contains noise, but does not contain any signal information. Then, a modified noise subspace projector is formed using the generalized eigenvectors of the noise and measured-data covariance matrices. The MUSIC localizer is calculated using this noise subspace projector and the noise covariance matrix. The results from a computer simulation have verified the effectiveness of the method. The method was then applied to source estimation for auditory-evoked fields elicited by syllable speech sounds. The results strongly suggest the method's effectiveness in removing the influence of background activity.

Reconstructing spatio-temporal activities of neural sources using an MEG vector beamformer technique.

We have developed a method suitable for reconstructing spatio-temporal activities of neural sources by using magnetoencephalogram (MEG) data. The method extends the adaptive beamformer technique originally proposed by Borgiotti and Kaplan to incorporate the vector beamformer formulation in which a set of three weight vectors are used to detect the source activity in three orthogonal directions. The weight vectors of the vector-extended version of the Borgiotti-Kaplan beamformer are then projected onto the signal subspace of the measurement covariance matrix to obtain the final form of the proposed beamformer's weight vectors. Our numerical experiments show that both spatial resolution and output signal-to-noise ratio of the proposed beamformer are significantly higher than those of the minimum-variance-based vector beamformer used in previous investigations. We also applied the proposed beamformer to two sets of auditory-evoked MEG data, and the results clearly demonstrated the method's capability of reconstructing spatio-temporal activities of neural sources.

MEG spatio-temporal analysis using a covariance matrix calculated from nonaveraged multiple-epoch data.

We propose a magnetoencephalographic (MEG) spatio-temporal analysis in which the measurement-covariance matrix is calculated using nonaveraged multiple epoch data. The proposed analysis has two advantages. First, a very narrow time window can be used for the source estimation. Second, accurate localization is possible even when the source activation has a time jitter. Experiments using auditory evoked MEG data clearly demonstrate these advantages.

MEG covariance difference analysis: a method to extract target source activities by using task and control measurements.

A method is proposed for extracting target dipolesource activities from two sets of evoked magnetoencephalographic (MEG) data, one measured using task stimuli and the other using control stimuli. The difference matrix between the two covariance matrices obtained from these two measurements is calculated, and a procedure similar to the MEG-multiple signal classification (MUSIC) algorithm is applied to this difference matrix to extract the target dipole-source configuration. This configuration corresponds to the source-configuration difference between the two measurements. Computer simulation verified the validity of the proposed method. The method was applied to actual evoked-field data obtained from simulated task-and-control experiments. In these measurements, a combination of auditory and somatosensory stimuli was used as the task stimulus and the somatosensory stimulus alone was used as the control stimulus. The proposed covariance difference analysis successfully extracted the target auditory source and eliminated the disturbance from the somatosensory sources.

Estimating neural sources from each time-frequency component of magnetoencephalographic data.

We have developed a method that incorporates the time-frequency characteristics of neural sources into magnetoencephalographic (MEG) source estimation. This method, referred to as the time-frequency multiple-signal-classification algorithm, allows the locations of neural sources to be estimated from any time-frequency region of interest. In this paper, we formulate the method based on the most general form of the quadratic time-frequency representations. We then apply it to two kinds of nonstationary MEG data: gamma-band (frequency range between 30-100 Hz) auditory activity data and spontaneous MEG data. Our method successfully detected the gamma-band source slightly medial to the N1m source location. The method was able to selectively localize sources for alpha-rhythm bursts at different locations. It also detected the mu-rhythm source from the alpha-rhythm-dominant MEG data that was measured with the subject's eyes closed. The results of these applications validate the effectiveness of the time-frequency MUSIC algorithm for selectively localizing sources having different time-frequency signatures.