Making sense of music: Insights from neurophysiology and connectivity analyses in naturalistic listening conditions.

According to the latest frameworks, auditory perception and memory involve the constant prediction of future sound events by the brain, based on the continuous extraction of feature regularities from the environment. The neural hierarchical mechanisms for predictive processes in perception and memory for sounds are typically studied in relation to simple acoustic features in isolated sounds or sound patterns inserted in highly certain contexts. Such studies have identified reliable prediction formation and error signals, e.g., the N100 or the mismatch negativity (MMN) evoked responses. In real life, though, individuals often face situations in which uncertainty prevails and where making sense of sounds becomes a hard challenge. In music, not only deviations from predictions are masterly set up by composers to induce emotions but sometimes the sheer uncertainty of sound scenes is exploited for aesthetic purposes, especially in compositional styles such as Western atonal classical music. In very recent magnetoencephalography (MEG) and electroencephalography (EEG) studies, experimental and technical advances in stimulation paradigms and analysis approaches have permitted the identification of prediction-error responses from highly uncertain, atonal contexts and the extraction of prediction-related responses from real, continuous music. Moreover, functional connectivity analyses revealed the emergence of cortico-hippocampal interactions during the formation of auditory memories for more predictable vs. less predictable patterns. These findings contribute to understanding the general brain mechanisms that enable us to predict even highly uncertain sound environments and to possibly make sense of and appreciate even atonal music.

Age differences in central auditory system responses to naturalistic music.

Aging influences the central auditory system leading to difficulties in the decoding and understanding of overlapping sound signals, such as speech in noise or polyphonic music. Studies on central auditory system evoked responses (ERs) have found in older compared to young listeners increased amplitudes (less inhibition) of the P1 and N1 and decreased amplitudes of the P2, mismatch negativity (MMN), and P3a responses. While preceding research has focused on simplified auditory stimuli, we here tested whether the previously observed age-related differences could be replicated with sounds embedded in medium and highly naturalistic musical contexts. Older (age 55-77 years) and younger adults (age 21-31 years) listened to medium naturalistic (synthesized melody) and highly naturalistic (studio recording of a music piece) stimuli. For the medium naturalistic music, the age group differences on the P1, N1, P2, MMN, and P3a amplitudes were all replicated. The age group differences, however, appeared reduced with the highly compared to the medium naturalistic music. The finding of lower P2 amplitude in older than young was replicated for slow event rates (0.3-2.9 Hz) in the highly naturalistic music. Moreover, the ER latencies suggested a gradual slowing of the auditory processing time course for highly compared to medium naturalistic stimuli irrespective of age. These results support that age-related differences on ERs can partly be observed with naturalistic stimuli. This opens new avenues for including naturalistic stimuli in the investigation of age-related central auditory system disorders.

Magnetoencephalography recordings reveal the spatiotemporal dynamics of recognition memory for complex versus simple auditory sequences.

Auditory recognition is a crucial cognitive process that relies on the organization of single elements over time. However, little is known about the spatiotemporal dynamics underlying the conscious recognition of auditory sequences varying in complexity. To study this, we asked 71 participants to learn and recognize simple tonal musical sequences and matched complex atonal sequences while their brain activity was recorded using magnetoencephalography (MEG). Results reveal qualitative changes in neural activity dependent on stimulus complexity: recognition of tonal sequences engages hippocampal and cingulate areas, whereas recognition of atonal sequences mainly activates the auditory processing network. Our findings reveal the involvement of a cortico-subcortical brain network for auditory recognition and support the idea that stimulus complexity qualitatively alters the neural pathways of recognition memory.

Consonance and dissonance perception. A critical review of the historical sources, multidisciplinary findings, and main hypotheses.

Revealed more than two millennia ago by Pythagoras, consonance and dissonance (C/D) are foundational concepts in music theory, perception, and aesthetics. The search for the biological, acoustical, and cultural factors that affect C/D perception has resulted in descriptive accounts inspired by arithmetic, musicological, psychoacoustical or neurobiological frameworks without reaching a consensus. Here, we review the key historical sources and modern multidisciplinary findings on C/D and integrate them into three main hypotheses: the vocal similarity hypothesis (VSH), the psychocultural hypothesis (PH), and the sensorimotor hypothesis (SH). By illustrating the hypotheses-related findings, we highlight their major conceptual, methodological, and terminological shortcomings. Trying to provide a unitary framework for C/D understanding, we put together multidisciplinary research on human and animal vocalizations, which converges to suggest that auditory roughness is associated with distress/danger and, therefore, elicits defensive behavioral reactions and neural responses that indicate aversion. We therefore stress the primacy of vocality and roughness as key factors in the explanation of C/D phenomenon, and we explore the (neuro)biological underpinnings of the attraction-aversion mechanisms that are triggered by C/D stimuli. Based on the reviewed evidence, while the aversive nature of dissonance appears as solidly rooted in the multidisciplinary findings, the attractive nature of consonance remains a somewhat speculative claim that needs further investigation. Finally, we outline future directions for empirical research in C/D, especially regarding cross-modal and cross-cultural approaches.

Enhanced mismatch negativity in harmonic compared with inharmonic sounds.

Many natural sounds have frequency spectra composed of integer multiples of a fundamental frequency. This property, known as harmonicity, plays an important role in auditory information processing. However, the extent to which harmonicity influences the processing of sound features beyond pitch is still unclear. This is interesting because harmonic sounds have lower information entropy than inharmonic sounds. According to predictive processing accounts of perception, this property could produce more salient neural responses due to the brain's weighting of sensory signals according to their uncertainty. In the present study, we used electroencephalography to investigate brain responses to harmonic and inharmonic sounds commonly occurring in music: Piano tones and hi-hat cymbal sounds. In a multifeature oddball paradigm, we measured mismatch negativity (MMN) and P3a responses to timbre, intensity, and location deviants in listeners with and without congenital amusia-an impairment of pitch processing. As hypothesized, we observed larger amplitudes and earlier latencies (for both MMN and P3a) in harmonic compared with inharmonic sounds. These harmonicity effects were modulated by sound feature. Moreover, the difference in P3a latency between harmonic and inharmonic sounds was larger for controls than amusics. We propose an explanation of these results based on predictive coding and discuss the relationship between harmonicity, information entropy, and precision weighting of prediction errors.

Musical prediction error responses similarly reduced by predictive uncertainty in musicians and non-musicians.

Auditory prediction error responses elicited by surprising sounds can be reliably recorded with musical stimuli that are more complex and realistic than those typically employed in EEG or MEG oddball paradigms. However, these responses are reduced as the predictive uncertainty of the stimuli increases. In this study, we investigate whether this effect is modulated by musical expertise. Magnetic mismatch negativity (MMNm) responses were recorded from 26 musicians and 24 non-musicians while they listened to low- and high-uncertainty melodic sequences in a musical multi-feature paradigm that included pitch, slide, intensity and timbre deviants. When compared to non-musicians, musically trained participants had significantly larger pitch and slide MMNm responses. However, both groups showed comparable reductions in pitch and slide MMNm amplitudes in the high-uncertainty condition compared with the low-uncertainty condition. In a separate, behavioural deviance detection experiment, musicians were more accurate and confident about their responses than non-musicians, but deviance detection in both groups was similarly affected by the uncertainty of the melodies. In both experiments, the interaction between uncertainty and expertise was not significant, suggesting that the effect is comparable in both groups. Consequently, our results replicate the modulatory effect of predictive uncertainty on prediction error; show that it is present across different types of listeners; and suggest that expertise-related and stimulus-driven modulations of predictive precision are dissociable and independent.

The chronnectome of musical beat.

Keeping time is fundamental for our everyday existence. Various isochronous activities, such as locomotion, require us to use internal timekeeping. This phenomenon comes into play also in other human pursuits such as dance and music. When listening to music, we spontaneously perceive and predict its beat. The process of beat perception comprises both beat inference and beat maintenance, their relative importance depending on the salience of beat in the music. To study functional connectivity associated with these processes in a naturalistic situation, we used functional magnetic resonance imaging to measure brain responses of participants while they were listening to a piece of music containing strong contrasts in beat salience. Subsequently, we utilized dynamic graph analysis and psychophysiological interactions (PPI) analysis in connection with computational modelling of beat salience to investigate how functional connectivity manifests these processes. As the main effect, correlation analyses between the obtained dynamic graph measures and the beat salience measure revealed increased centrality in auditory-motor cortices, cerebellum, and extrastriate visual areas during low beat salience, whereas regions of the default mode- and central executive networks displayed high centrality during high beat salience. PPI analyses revealed partial dissociation of functional networks belonging to this pathway indicating complementary neural mechanisms crucial in beat inference and maintenance, processes pivotal for extracting and predicting temporal regularities in our environment.

Reduced prediction error responses in high-as compared to low-uncertainty musical contexts.

Theories of predictive processing propose that prediction error responses are modulated by the certainty of the predictive model or precision. While there is some evidence for this phenomenon in the visual and, to a lesser extent, the auditory modality, little is known about whether it operates in the complex auditory contexts of daily life. Here, we examined how prediction error responses behave in a more complex and ecologically valid auditory context than those typically studied. We created musical tone sequences with different degrees of pitch uncertainty to manipulate the precision of participants' auditory expectations. Magnetoencephalography was used to measure the magnetic counterpart of the mismatch negativity (MMNm) as a neural marker of prediction error in a multi-feature paradigm. Pitch, slide, intensity and timbre deviants were included. We compared high-entropy stimuli, consisting of a set of non-repetitive melodies, with low-entropy stimuli consisting of a simple, repetitive pitch pattern. Pitch entropy was quantitatively assessed with an information-theoretic model of auditory expectation. We found a reduction in pitch and slide MMNm amplitudes in the high-entropy as compared to the low-entropy context. No significant differences were found for intensity and timbre MMNm amplitudes. Furthermore, in a separate behavioral experiment investigating the detection of pitch deviants, similar decreases were found for accuracy measures in response to more fine-grained increases in pitch entropy. Our results are consistent with a precision modulation of auditory prediction error in a musical context, and suggest that this effect is specific to features that depend on the manipulated dimension-pitch information, in this case.

Fractionating auditory priors: A neural dissociation between active and passive experience of musical sounds.

Learning, attention and action play a crucial role in determining how stimulus predictions are formed, stored, and updated. Years-long experience with the specific repertoires of sounds of one or more musical styles is what characterizes professional musicians. Here we contrasted active experience with sounds, namely long-lasting motor practice, theoretical study and engaged listening to the acoustic features characterizing a musical style of choice in professional musicians with mainly passive experience of sounds in laypersons. We hypothesized that long-term active experience of sounds would influence the neural predictions of the stylistic features in professional musicians in a distinct way from the mainly passive experience of sounds in laypersons. Participants with different musical backgrounds were recruited: professional jazz and classical musicians, amateur musicians and non-musicians. They were presented with a musical multi-feature paradigm eliciting mismatch negativity (MMN), a prediction error signal to changes in six sound features for only 12 minutes of electroencephalography (EEG) and magnetoencephalography (MEG) recordings. We observed a generally larger MMN amplitudes-indicative of stronger automatic neural signals to violated priors-in jazz musicians (but not in classical musicians) as compared to non-musicians and amateurs. The specific MMN enhancements were found for spectral features (timbre, pitch, slide) and sound intensity. In participants who were not musicians, the higher preference for jazz music was associated with reduced MMN to pitch slide (a feature common in jazz music style). Our results suggest that long-lasting, active experience of a musical style is associated with accurate neural priors for the sound features of the preferred style, in contrast to passive listening.

Weighting of neural prediction error by rhythmic complexity: A predictive coding account using mismatch negativity.

The human brain's ability to extract and encode temporal regularities and to predict the timing of upcoming events is critical for music and speech perception. This work addresses how these mechanisms deal with different levels of temporal complexity, here the number of distinct durations in rhythmic patterns. We use electroencephalography (EEG) to relate the mismatch negativity (MMN), a proxy of neural prediction error, to a measure of information content of rhythmic sequences, the Shannon entropy. Within each of three conditions, participants listened to repeatedly presented standard rhythms of five tones (four inter-onset intervals) and of a given level of entropy: zero (isochronous), medium entropy (two distinct interval durations), or high entropy (four distinct interval durations). Occasionally, the fourth tone was moved forward in time that is it occurred 100 ms (small deviation) or 300 ms early (large deviation). According to the predictive coding framework, high-entropy stimuli are more difficult to model for the brain, resulting in less confident predictions and yielding smaller prediction errors for deviant sounds. Our results support this hypothesis, showing a gradual decrease in MMN amplitude as a function of entropy, but only for small timing deviants. For large timing deviants, in contrast, a modulation of activity in the opposite direction was observed for the earlier N1 component, known to also be sensitive to sudden changes in directed attention. Our results suggest the existence of a fine-grained neural mechanism that weights neural prediction error to the complexity of rhythms and that mostly manifests in the absence of directed attention.

From random to regular: neural constraints on the emergence of isochronous rhythm during cultural transmission.

A core design feature of human communication systems and expressive behaviours is their temporal organization. The cultural evolutionary origins of this feature remain unclear. Here, we test the hypothesis that regularities in the temporal organization of signalling sequences arise in the course of cultural transmission as adaptations to aspects of cortical function. We conducted two experiments on the transmission of rhythms associated with affective meanings, focusing on one of the most widespread forms of regularity in language and music: isochronicity. In the first experiment, we investigated how isochronous rhythmic regularities emerge and change in multigenerational signalling games, where the receiver (learner) in a game becomes the sender (transmitter) in the next game. We show that signalling sequences tend to become rhythmically more isochronous as they are transmitted across generations. In the second experiment, we combined electroencephalography (EEG) and two-player signalling games over 2 successive days. We show that rhythmic regularization of sequences can be predicted based on the latencies of the mismatch negativity response in a temporal oddball paradigm. These results suggest that forms of isochronicity in communication systems originate in neural constraints on information processing, which may be expressed and amplified in the course of cultural transmission.

Decoding Musical Training from Dynamic Processing of Musical Features in the Brain.

Pattern recognition on neural activations from naturalistic music listening has been successful at predicting neural responses of listeners from musical features, and vice versa. Inter-subject differences in the decoding accuracies have arisen partly from musical training that has widely recognized structural and functional effects on the brain. We propose and evaluate a decoding approach aimed at predicting the musicianship class of an individual listener from dynamic neural processing of musical features. Whole brain functional magnetic resonance imaging (fMRI) data was acquired from musicians and nonmusicians during listening of three musical pieces from different genres. Six musical features, representing low-level (timbre) and high-level (rhythm and tonality) aspects of music perception, were computed from the acoustic signals, and classification into musicians and nonmusicians was performed on the musical feature and parcellated fMRI time series. Cross-validated classification accuracy reached 77% with nine regions, comprising frontal and temporal cortical regions, caudate nucleus, and cingulate gyrus. The processing of high-level musical features at right superior temporal gyrus was most influenced by listeners' musical training. The study demonstrates the feasibility to decode musicianship from how individual brains listen to music, attaining accuracy comparable to current results from automated clinical diagnosis of neurological and psychological disorders.

Connectivity patterns during music listening: Evidence for action-based processing in musicians.

Musical expertise is visible both in the morphology and functionality of the brain. Recent research indicates that functional integration between multi-sensory, somato-motor, default-mode (DMN), and salience (SN) networks of the brain differentiates musicians from non-musicians during resting state. Here, we aimed at determining whether brain networks differentially exchange information in musicians as opposed to non-musicians during naturalistic music listening. Whole-brain graph-theory analyses were performed on participants' fMRI responses. Group-level differences revealed that musicians' primary hubs comprised cerebral and cerebellar sensorimotor regions whereas non-musicians' dominant hubs encompassed DMN-related regions. Community structure analyses of the key hubs revealed greater integration of motor and somatosensory homunculi representing the upper limbs and torso in musicians. Furthermore, musicians who started training at an earlier age exhibited greater centrality in the auditory cortex, and areas related to top-down processes, attention, emotion, somatosensory processing, and non-verbal processing of speech. We here reveal how brain networks organize themselves in a naturalistic music listening situation wherein musicians automatically engage neural networks that are action-based while non-musicians use those that are perception-based to process an incoming auditory stream. Hum Brain Mapp 38:2955-2970, 2017. © 2017 Wiley Periodicals, Inc.

A window into the brain mechanisms associated with noise sensitivity.

Noise sensitive individuals are more likely to experience negative emotions from unwanted sounds and they show greater susceptibility to adverse effects of noise on health. Noise sensitivity does not originate from dysfunctions of the peripheral auditory system, and it is thus far unknown whether and how it relates to abnormalities of auditory processing in the central nervous system. We conducted a combined electroencephalography and magnetoencephalography (M/EEG) study to measure neural sound feature processing in the central auditory system in relation to the individual noise sensitivity. Our results show that high noise sensitivity is associated with altered sound feature encoding and attenuated discrimination of sound noisiness in the auditory cortex. This finding makes a step towards objective measures of noise sensitivity instead of self-evaluation questionnaires and the development of strategies to prevent negative effects of noise on the susceptible population.

Comparing the Performance of Popular MEG/EEG Artifact Correction Methods in an Evoked-Response Study.

We here compared results achieved by applying popular methods for reducing artifacts in magnetoencephalography (MEG) and electroencephalography (EEG) recordings of the auditory evoked Mismatch Negativity (MMN) responses in healthy adult subjects. We compared the Signal Space Separation (SSS) and temporal SSS (tSSS) methods for reducing noise from external and nearby sources. Our results showed that tSSS reduces the interference level more reliably than plain SSS, particularly for MEG gradiometers, also for healthy subjects not wearing strongly interfering magnetic material. Therefore, tSSS is recommended over SSS. Furthermore, we found that better artifact correction is achieved by applying Independent Component Analysis (ICA) in comparison to Signal Space Projection (SSP). Although SSP reduces the baseline noise level more than ICA, SSP also significantly reduces the signal-slightly more than it reduces the artifacts interfering with the signal. However, ICA also adds noise, or correction errors, to the waveform when the signal-to-noise ratio (SNR) in the original data is relatively low-in particular to EEG and to MEG magnetometer data. In conclusion, ICA is recommended over SSP, but one should be careful when applying ICA to reduce artifacts on neurophysiological data with relatively low SNR.

Hidden sources of joy, fear, and sadness: Explicit versus implicit neural processing of musical emotions.

Music is often used to regulate emotions and mood. Typically, music conveys and induces emotions even when one does not attend to them. Studies on the neural substrates of musical emotions have, however, only examined brain activity when subjects have focused on the emotional content of the music. Here we address with functional magnetic resonance imaging (fMRI) the neural processing of happy, sad, and fearful music with a paradigm in which 56 subjects were instructed to either classify the emotions (explicit condition) or pay attention to the number of instruments playing (implicit condition) in 4-s music clips. In the implicit vs. explicit condition, stimuli activated bilaterally the inferior parietal lobule, premotor cortex, caudate, and ventromedial frontal areas. The cortical dorsomedial prefrontal and occipital areas activated during explicit processing were those previously shown to be associated with the cognitive processing of music and emotion recognition and regulation. Moreover, happiness in music was associated with activity in the bilateral auditory cortex, left parahippocampal gyrus, and supplementary motor area, whereas the negative emotions of sadness and fear corresponded with activation of the left anterior cingulate and middle frontal gyrus and down-regulation of the orbitofrontal cortex. Our study demonstrates for the first time in healthy subjects the neural underpinnings of the implicit processing of brief musical emotions, particularly in frontoparietal, dorsolateral prefrontal, and striatal areas of the brain.

Comprehensive auditory discrimination profiles recorded with a fast parametric musical multi-feature mismatch negativity paradigm.

OBJECTIVE: Mismatch negativity (MMN), a component of the auditory event-related potential (ERP) in response to auditory-expectancy violation, is sensitive to central auditory processing deficits associated with several clinical conditions and to auditory skills deriving from musical expertise. This sensitivity is more evident for stimuli integrated in complex sound contexts. This study tested whether increasing magnitudes of deviation (levels) entail increasing MMN amplitude (or decreasing latency), aiming to create a balanced version of the musical multi-feature paradigm towards measurement of extensive auditory discrimination profiles in auditory expertise or deficits. METHODS: Using electroencephalography, we measured MMNs in healthy young adults to six types of sound feature change (pitch, timbre, location, intensity, slide and rhythm) at three different magnitudes of deviation, embedded in a music-sounding context. We also behaviourally assessed the individual musical aptitude using the Musical Ear Test (MET). RESULTS: 16 of 18 sound feature changes elicited significant MMNs. For pitch, intensity, location, and slide, the MMN amplitude increased with increasing magnitude of feature change. We observed a ceiling effect for rhythm, and a floor effect for timbre. The slide MMN amplitude correlated positively with MET melody score and negatively with MET rhythm score. CONCLUSIONS: This novel paradigm provides an extensive, objective measure of auditory discrimination profile for different sound features embedded in a complex sound context. SIGNIFICANCE: The paradigm can be adopted to study the neurophysiology of individuals with music processing difficulties or with special musical skills, and may be a useful tool for investigating development, plasticity, and deficits of auditory processing.

The reliability of continuous brain responses during naturalistic listening to music.

Low-level (timbral) and high-level (tonal and rhythmical) musical features during continuous listening to music, studied by functional magnetic resonance imaging (fMRI), have been shown to elicit large-scale responses in cognitive, motor, and limbic brain networks. Using a similar methodological approach and a similar group of participants, we aimed to study the replicability of previous findings. Participants' fMRI responses during continuous listening of a tango Nuevo piece were correlated voxelwise against the time series of a set of perceptually validated musical features computationally extracted from the music. The replicability of previous results and the present study was assessed by two approaches: (a) correlating the respective activation maps, and (b) computing the overlap of active voxels between datasets at variable levels of ranked significance. Activity elicited by timbral features was better replicable than activity elicited by tonal and rhythmical ones. These results indicate more reliable processing mechanisms for low-level musical features as compared to more high-level features. The processing of such high-level features is probably more sensitive to the state and traits of the listeners, as well as of their background in music.

The association of noise sensitivity with music listening, training, and aptitude.

After intensive, long-term musical training, the auditory system of a musician is specifically tuned to perceive musical sounds. We wished to find out whether a musician's auditory system also develops increased sensitivity to any sound of everyday life, experiencing them as noise. For this purpose, an online survey, including questionnaires on noise sensitivity, musical background, and listening tests for assessing musical aptitude, was administered to 197 participants in Finland and Italy. Subjective noise sensitivity (assessed with the Weinstein's Noise Sensitivity Scale) was analyzed for associations with musicianship, musical aptitude, weekly time spent listening to music, and the importance of music in each person's life (or music importance). Subjects were divided into three groups according to their musical expertise: Nonmusicians (N = 103), amateur musicians (N = 44), and professional musicians (N = 50). The results showed that noise sensitivity did not depend on musical expertise or performance on musicality tests or the amount of active (attentive) listening to music. In contrast, it was associated with daily passive listening to music, so that individuals with higher noise sensitivity spent less time in passive (background) listening to music than those with lower sensitivity to noise. Furthermore, noise-sensitive respondents rated music as less important in their life than did individuals with lower sensitivity to noise. The results demonstrate that the special sensitivity of the auditory system derived from musical training does not lead to increased irritability from unwanted sounds. However, the disposition to tolerate contingent musical backgrounds in everyday life depends on the individual's noise sensitivity.