Cognitive and sensory expectations independently shape musical expectancy and pleasure.

Expectation is crucial for our enjoyment of music, yet the underlying generative mechanisms remain unclear. While sensory models derive predictions based on local acoustic information in the auditory signal, cognitive models assume abstract knowledge of music structure acquired over the long term. To evaluate these two contrasting mechanisms, we compared simulations from four computational models of musical expectancy against subjective expectancy and pleasantness ratings of over 1000 chords sampled from 739 US Billboard pop songs. Bayesian model comparison revealed that listeners' expectancy and pleasantness ratings were predicted by the independent, non-overlapping, contributions of cognitive and sensory expectations. Furthermore, cognitive expectations explained over twice the variance in listeners' perceived surprise compared to sensory expectations, suggesting a larger relative importance of long-term representations of music structure over short-term sensory-acoustic information in musical expectancy. Our results thus emphasize the distinct, albeit complementary, roles of cognitive and sensory expectations in shaping musical pleasure, and suggest that this expectancy-driven mechanism depends on musical information represented at different levels of abstraction along the neural hierarchy. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Longitudinal trajectories of electrophysiological mismatch responses in infant speech discrimination differ across speech features.

Infants rapidly advance in their speech perception, electrophysiologically reflected in the transition from an immature, positive-going to an adult-like, negative-going mismatch response (MMR) to auditory deviancy. Although the MMR is a common tool to study speech perception development, it is not yet completely understood how different speech contrasts affect the MMR's characteristics across development. Thus, a systematic longitudinal investigation of the MMR's maturation depending on speech contrast is necessary. We here longitudinally explored the maturation of the infant MMR to four critical speech contrasts: consonant, vowel, vowel-length, and pitch. MMRs were obtained when infants (n = 58) were 2, 6 and 10 months old. To evaluate the maturational trajectory of MMRs, we applied second-order latent growth curve models. Results showed positive-going MMR amplitudes to all speech contrasts across all assessment points that decreased over time towards an adult-like negativity. Notably, the developmental trajectories of speech contrasts differed, implying that infant speech perception matures with different rates and trajectories throughout the first year, depending on the studied auditory feature. Our results suggest that stimulus-dependent maturational trajectories need to be considered when drawing conclusions about infant speech perception development reflected by the infant MMR.

The topographical organization of motor processing: An ALE meta-analysis on six action domains and the relevance of Broca's region.

Action is a cover term used to refer to a large set of motor processes differing in domain specificities (e.g. execution or observation). Here we review neuroimaging evidence on action processing (N = 416; Subjects = 5912) using quantitative Activation Likelihood Estimation (ALE) and Meta-Analytic Connectivity Modeling (MACM) approaches to delineate the functional specificities of six domains: (1) Action Execution, (2) Action Imitation, (3) Motor Imagery, (4) Action Observation, (5) Motor Learning, (6) Motor Preparation. Our results show distinct functional patterns for the different domains with convergence in posterior BA44 (pBA44) for execution, imitation and imagery processing. The functional connectivity network seeding in the motor-based localized cluster of pBA44 differs from the connectivity network seeding in the (language-related) anterior BA44. The two networks implement distinct cognitive functions. We propose that the motor-related network encompassing pBA44 is recruited when processing movements requiring a mental representation of the action itself.

Mathematical expertise modulates the architecture of dorsal and cortico-thalamic white matter tracts.

To what extent are levels of cognitive expertise reflected in differential structural connectivity of the brain? We addressed this question by analyzing the white matter brain structure of experts (mathematicians) versus non-experts (non-mathematicians) using probabilistic tractography. Having mathematicians and non-mathematicians as participant groups enabled us to directly compare profiles of structural connectivity arising from individual levels of expertise in mathematics. Tracking from functional seed regions activated during the processing of complex arithmetic formulas revealed an involvement of various fiber bundles such the inferior fronto-occipital fascicle, arcuate fasciculus/superior longitudinal fasciculus (AF/SLF), cross-hemispheric connections of frontal lobe areas through the corpus callosum and cortico-subcortical connectivity via the bilateral thalamic radiation. With the aim of investigating expertise-dependent structural connectivity, the streamline density was correlated with the level of expertise, defined by automaticity of processing complex mathematics. The results showed that structural integrity of the AF/SLF was higher in individuals with higher automaticity, while stronger cortico-thalamic connectivity was associated with lower levels of automaticity. Therefore, we suggest that expertise in the domain of mathematics is reflected in plastic changes of the brain's white matter structure, possibly reflecting a general principle of cognitive expertise.

Evolutionary origins of non-adjacent sequence processing in primate brain potentials.

There is considerable interest in understanding the ontogeny and phylogeny of the human language system, yet, neurobiological work at the interface of both fields is absent. Syntactic processes in language build on sensory processing and sequencing capabilities on the side of the receiver. While we better understand language-related ontogenetic changes in the human brain, it remains a mystery how neurobiological processes at specific human development stages compare with those in phylogenetically closely related species. To address this knowledge gap, we measured EEG event-related potentials (ERPs) in two macaque monkeys using a paradigm developed to evaluate human infant and adult brain potentials associated with the processing of non-adjacent ordering relationships in sequences of syllable triplets. Frequent standard triplet sequences were interspersed with infrequent voice pitch or non-adjacent rule deviants. Monkey ERPs show early pitch and rule deviant mismatch responses that are strikingly similar to those previously reported in human infants. This stands in contrast to adults' later ERP responses for rule deviants. The results reveal how non-adjacent sequence ordering relationships are processed in the primate brain and provide evidence for evolutionarily conserved neurophysiological effects, some of which are remarkably like those seen at an early human developmental stage.

Classifying song and speech: effects of focal temporal lesions and musical disorder.

Song and speech represent two auditory categories the brain usually classifies fairly easily. Functionally, this classification ability may depend to a great extent on characteristic features of pitch patterns present in song melody and speech prosody. Anatomically, the temporal lobe (TL) has been discussed as playing a prominent role in the processing of both. Here we tested individuals with congenital amusia and patients with unilateral left and right TL lesions in their ability to categorize song and speech. In a forced-choice paradigm, specifically designed auditory stimuli representing sung, spoken and "ambiguous" stimuli (being perceived as "halfway between" song and speech), had to be classified as either "song" or "speech". Congenital amusics and TL patients, contrary to controls, exhibited a surprising bias to classifying the ambiguous stimuli as "song" despite their apparent deficit to correctly process features typical for song. This response bias possibly reflects a strategy where, based on available context information (here: forced choice for either speech or song), classification of non-processable items may be achieved through elimination of processable classes. This speech-based strategy masks the pitch processing deficit in congenital amusics and TL lesion patients.

Neural correlates of prosodic boundary perception in German preschoolers: If pause is present, pitch can go.

Children׳s perception of prosodic phrasing provides a head start into the discovery of speech structure. Based on the close prosody-syntax correspondence, children can infer the underlying syntactic structure from the acoustic modulations of prosodic boundaries, typically consisting of co-occurring pitch changes, preboundary lengthening, and pausing. Previous electrophysiological studies revealed that listeners are to some degree flexible in the detection of major prosodic boundaries that are not marked with all three of the suprasegmental cues. Adults and 6-year-olds still showed the brain response for prosodic boundary perception, the Closure Positive Shift (CPS), when pauses marking boundaries were deleted. In contrast, younger children at 3 years did not show this ability yet, but required pausing to complement the other boundary cues. Following the hypothesis that German weights duration cues more heavily than pitch cues, we here examined 3-year-olds׳ brain responses to prosodic phrasing, testing the role of boundary-related pitch changes. Results revealed that children at this age even showed the CPS in response to pitch-neutralized boundaries with only pausing and preboundary lengthening being present. These results indicate differential roles of acoustic cues in boundary perception, with a preferential reliance on duration cues over pitch changes in 3-year-olds. This preference likely results from the characteristics of the German intonation system and furthers the discussion of cross-linguistic differences in the weighting of prosodic boundary cues.

Syntax in Action Has Priority over Movement Selection in Piano Playing: An ERP Study.

Complex human behavior is hierarchically organized. Whether or not syntax plays a role in this organization is currently under debate. The present ERP study uses piano performance to isolate syntactic operations in action planning and to demonstrate their priority over nonsyntactic levels of movement selection. Expert pianists were asked to execute chord progressions on a mute keyboard by copying the posture of a performing model hand shown in sequences of photos. We manipulated the final chord of each sequence in terms of Syntax (congruent/incongruent keys) and Manner (conventional/unconventional fingering), as well as the strength of its predictability by varying the length of the Context (five-chord/two-chord progressions). The production of syntactically incongruent compared to congruent chords showed a response delay that was larger in the long compared to the short context. This behavioral effect was accompanied by a centroparietal negativity in the long but not in the short context, suggesting that a syntax-based motor plan was prepared ahead. Conversely, the execution of the unconventional manner was not delayed as a function of Context and elicited an opposite electrophysiological pattern (a posterior positivity). The current data support the hypothesis that motor plans operate at the level of musical syntax and are incrementally translated to lower levels of movement selection.

Functional network mirrored in the prefrontal cortex, caudate nucleus, and thalamus: high-resolution functional imaging and structural connectivity.

Despite myriads of studies on a parallel organization of cortico-striatal-thalamo-cortical loops, direct evidence of this has been lacking for the healthy human brain. Here, we scrutinize the functional specificity of the cortico-subcortical loops depending on varying levels of cognitive hierarchy as well as their structural connectivity with high-resolution fMRI and diffusion-weighted MRI (dMRI) at 7 tesla. Three levels of cognitive hierarchy were implemented in two domains: second language and nonlanguage. In fMRI, for the higher level, activations were found in the ventroanterior portion of the prefrontal cortex (PFC), the head of the caudate nucleus (CN), and the ventral anterior nucleus (VA) in the thalamus. Conversely, for the lower level, activations were located in the posterior region of the PFC, the body of the CN, and the medial dorsal nucleus (MD) in the thalamus. This gradient pattern of activations was furthermore shown to be tenable by the parallel connectivity in dMRI tractography connecting the anterior regions of the PFC with the head of the CN and the VA in the thalamus, whereas the posterior activations of the PFC were linked to the body of the CN and the MD in the thalamus. This is the first human in vivo study combining fMRI and dMRI showing that the functional specificity is mirrored within the cortico-subcortical loop substantiated by parallel networks.

Accentuate or repeat? Brain signatures of developmental periods in infant word recognition.

Language acquisition has long been discussed as an interaction between biological preconditions and environmental input. This general interaction seems particularly salient in lexical acquisition, where infants are already able to detect unknown words in sentences at 7 months of age, guided by phonological and statistical information in the speech input. While this information results from the linguistic structure of a given language, infants also exploit situational information, such as speakers' additional word accentuation and word repetition. The current study investigated the developmental trajectory of infants' sensitivity to these two situational input cues in word recognition. Testing infants at 6, 9, and 12 months of age, we hypothesized that different age groups are differentially sensitive to accentuation and repetition. In a familiarization-test paradigm, event-related brain potentials (ERPs) revealed age-related differences in infants' word recognition as a function of situational input cues: at 6 months infants only recognized previously accentuated words, at 9 months both accentuation and repetition played a role, while at 12 months only repetition was effective. These developmental changes are suggested to result from infants' advancing linguistic experience and parallel auditory cortex maturation. Our data indicate very narrow and specific input-sensitive periods in infant word recognition, with accentuation being effective prior to repetition.

The role of pause as a prosodic boundary marker: language ERP studies in German 3- and 6-year-olds.

Spoken language is hierarchically structured into prosodic units divided by prosodic breaks. The largest prosodic breaks in an utterance are intonational phrase boundaries (IPBs), which are defined by three acoustic cues, namely, pitch change, preboundary lengthening, and pausing. Previous studies have revealed that the electrophysiological marker of IPB perception, the Closure Positive Shift (CPS), is established between 2 and 3 years of age. Here, we examined the neural activity underlying IPB perception in children by targeting their reliance on pausing; hypothesized to be a key boundary cue in German. To evaluate the role of pausing, we tested IPB perception without the boundary pause, but with pitch change and preboundary lengthening. We tested children at the age of 3 years, when the CPS in response to IPBs has just emerged, and at 6 years, when language abilities are further developed. Results revealed that 6-year-olds, but not 3-year-olds, show the CPS in response to IPBs without full prosodic marking. These results indicate developmental differences with respect to the role of pausing as a prosodic boundary cue in German. The correlation of children's IPB perception and their syntactic abilities further corroborates the close prosody-syntax interaction in children's advancing ability to process phrase structure.

Language learning without control: the role of the PFC.

Learning takes place throughout lifetime but differs in infants and adults because of the development of the PFC, a brain region responsible for cognitive control. To test this hypothesis, adults were investigated in a language learning paradigm under inhibitory, cathodal transcranial direct current stimulation over PFC. The experiment included a learning session interspersed with test phases and a test-only session. The stimulus material required the learning of grammatical dependencies between two elements in a novel language. In a parallel design, cathodal transcranial direct current stimulation over the left PFC, right PFC, or sham stimulation was applied during the learning session but not during the test-only session. Event-related brain potentials (ERPs) were recorded during both sessions. Whereas no ERP learning effects were observed during the learning session, different ERP learning effects as a function of prior stimulation type were found during the test-only session, although behavioral learning success was equal across conditions. With sham stimulation, the ERP learning effect was reflected in a centro-parietal N400-like negativity indicating lexical processes. Inhibitory stimulation over the left PFC, but not over the right PFC, led to a late positivity similar to that previously observed in prelinguistic infants indicating associative learning. The present data demonstrate that adults can learn with and without cognitive control using different learning mechanisms. In the presence of cognitive control, adult language learning is lexically guided, whereas it appears to be associative in nature when PFC control is downregulated.

Left parietal alpha enhancement during working memory-intensive sentence processing.

Both functional magnetic resonance imaging (fMRI) and event-related brain potential (ERP) studies have shown that verbal working memory plays an important role during sentence processing. There is growing evidence from outside of sentence processing that human alpha oscillations (7-13 Hz) play a critical role in working memory. This study aims to link this to the sentence processing domain. Time-frequency analyses and source localization were performed on electroencephalography (EEG) data that were recorded during the processing of auditorily presented sentences involving either a short or a long distance between an argument (subject or object) and the respective sentence-final verb. We reasoned that oscillatory activity in the alpha band should increase during sentences with longer argument-verb distances, since decreased temporal proximity should result in increased memory demands. When verbal working memory-intensive long-dependency sentences were compared to short-dependency sentences, a sustained oscillatory enhancement at 10 Hz was found during storage prior to the sentence-final verb, turning into a transient power increase in the beta band (13-20 Hz) at the sentence-final verb. The sources of the alpha oscillations were localized to bilaterally occipital and left parietal cortices. Only the source activity in the left parietal cortex was negatively correlated with verbal working memory abilities. These findings indicate that the parsimonious role of alpha oscillations in domain-general working memory can be extended to language, that is, sentence processing. We suggest that the function of left parietal cortex underlying verbal working memory storage during sentence processing is to inhibit the premature release of verbal information that will subsequently be integrated.

Preschool children's interpretation of object-initial sentences: neural correlates of their behavioral performance.

The acquisition of the function of case-marking is a key step in the development of sentence processing for German-speaking children since case-marking reveals the relations between sentential arguments. In this study, we investigated the development of the processing of case-marking and argument structures in children at 3, 4;6 and 6 years of age, as well as its processing in adults. Using EEG, we measured event-related potentials (ERPs) in response to object-initial compared to subject-initial German sentences including transitive verbs and case-marked noun phrases referring to animate arguments. We also tested children's behavioral competence in a sentence-picture matching task. Word order and case-marking were manipulated in German main clauses. Adults' behavioral performance was close to perfect and their ERPs revealed a negativity for the processing of the topicalized accusative marked noun phrase (NP1) and no effect for the second NP (NP2) in the object-initial structure. Children's behavioral data showed a significant above-chance outcome in the subject-initial condition for all age groups, but not for the object-initial condition. In contrast to adults, the ERPs of 3-year-olds showed a positivity at NP1, indicating difficulties in processing the non-canonical object-initial structures. Children at the age of 4;6 did not differ in the processing patterns of object-initial vs. subject-initial sentences at NP1 but showed a slight positivity at NP2. This positivity at NP2, which implies syntactic integration difficulties, is more pronounced in 6-year-olds but is absent in adults. At NP1, however, 6-year-olds show the same negativity as adults. In sum, the behavioral and electrophysiological findings demonstrate that children in each age group use different strategies, which are indicative of their developmental stage. While 3-year-olds merely detect differences in the two sentence structures without being able to use this information for sentence comprehension, 4;6-year-olds proceed to use mainly a word-order strategy, processing NP1 in both conditions in the same manner, which leads to processing difficulties upon detecting case-marking cues at NP2. At the age of 6, children are able to use case-marking cues for comprehension but still show enhanced effort for correct thematic-role assignment.

Neural correlates of perceptual narrowing in cross-species face-voice matching.

Integrating the multisensory features of talking faces is critical to learning and extracting coherent meaning from social signals. While we know much about the development of these capacities at the behavioral level, we know very little about the underlying neural processes. One prominent behavioral milestone of these capacities is the perceptual narrowing of face-voice matching, whereby young infants match faces and voices across species, but older infants do not. In the present study, we provide neurophysiological evidence for developmental decline in cross-species face-voice matching. We measured event-related brain potentials (ERPs) while 4- and 8-month-old infants watched and listened to congruent and incongruent audio-visual presentations of monkey vocalizations and humans mimicking monkey vocalizations. The ERP results indicated that younger infants distinguished between the congruent and the incongruent faces and voices regardless of species, whereas in older infants, the sensitivity to multisensory congruency was limited to the human face and voice. Furthermore, with development, visual and frontal brain processes and their functional connectivity became more sensitive to the congruence of human faces and voices relative to monkey faces and voices. Our data show the neural correlates of perceptual narrowing in face-voice matching and support the notion that postnatal experience with species identity is associated with neural changes in multisensory processing (Lewkowicz & Ghazanfar, 2009).

Linking ordering in Broca's area to storage in left temporo-parietal regions: the case of sentence processing.

In sentence processing, storage and ordering of the verb and its arguments (subject and object) are core tasks. Their cortical representation is a matter of ongoing debate, and it is unclear whether prefrontal activations in neuroimaging studies on sentence processing reflect the storage of arguments or their ordering. Moreover, it is unclear how storage during sentence processing relates to the neuroanatomy of storage outside the sentence processing domain. To tackle these questions, we crossed the factor "ordering" (subject-first vs. object-first German sentences) with the factor "storage" (one vs. four phrases intervene between the critical argument and the verb) in an auditory fMRI study. Ordering focally activated the left pars opercularis in Broca's area, while storage activated deep left temporo-parietal (TP) regions. Notably, left TP activation correlated with listener's digit span, while Broca's area activation did not. Furthermore, fractional anisotropy of listeners' left arcuate fasciculus/superior longitudinal fasciculus (AF/SLF) is shown to covary with the functional effect of increased storage demands at sites along the tract. Functionally, the results suggest that storage during sentence processing relies on TP regions, likely shared between sentence processing and other working memory-related tasks, while Broca's area appears as a distinct neural correlate of ordering. We conclude that the abstract notion of sentence processing can be captured by the interplay of concrete cognitive concepts such as ordering and storage.

Auditory perception and syntactic cognition: brain activity-based decoding within and across subjects.

The present magnetoencephalography study investigated whether the brain states of early syntactic and auditory-perceptual processes can be decoded from single-trial recordings with a multivariate pattern classification approach. In particular, it was investigated whether the early neural activation patterns in response to rule violations in basic auditory perception and in high cognitive processes (syntax) reflect a functional organization that largely generalizes across individuals or is subject-specific. On this account, subjects were auditorily presented with correct sentences, syntactically incorrect sentences, correct sentences including an interaural time difference change, and sentences containing both violations. For the analysis, brain state decoding was carried out within and across subjects with three pairwise classifications. Neural patterns elicited by each of the violation sentences were separately classified with the patterns elicited by the correct sentences. The results revealed the highest decoding accuracies over temporal cortex areas for all three classification types. Importantly, both the magnitude and the spatial distribution of decoding accuracies for the early neural patterns were very similar for within-subject and across-subject decoding. At the same time, across-subject decoding suggested a hemispheric bias, with the most consistent patterns in the left hemisphere. Thus, the present data show that not only auditory-perceptual processing brain states but also cognitive brain states of syntactic rule processing can be decoded from single-trial brain activations. Moreover, the findings indicate that the neural patterns in response to syntactic cognition and auditory perception reflect a functional organization that is highly consistent across individuals.

Dissociable neural imprints of perception and grammar in auditory functional imaging.

In language processing, the relative contribution of early sensory and higher cognitive brain areas is still an open issue. A recent controversial hypothesis proposes that sensory cortices show sensitivity to syntactic processes, whereas other studies suggest a wider neural network outside sensory regions. The goal of the current event-related fMRI study is to clarify the contribution of sensory cortices in auditory syntactic processing in a 2 × 2 design. Two-word utterances were presented auditorily and varied both in perceptual markedness (presence or absence of an overt word category marking "-t"), and in grammaticality (syntactically correct or incorrect). A multivariate pattern classification approach was applied to the data, flanked by conventional cognitive subtraction analyses. The combination of methods and the 2 × 2 design revealed a clear picture: The cognitive subtraction analysis found initial syntactic processing signatures in a neural network including the left IFG, the left aSTG, the left superior temporal sulcus (STS), as well as the right STS/STG. Classification of local multivariate patterns indicated the left-hemispheric regions in IFG, aSTG, and STS to be more syntax-specific than the right-hemispheric regions. Importantly, auditory sensory cortices were only sensitive to the overt perceptual marking, but not to the grammaticality, speaking against syntax-inflicted sensory cortex modulations. Instead, our data provide clear evidence for a distinction between regions involved in pure perceptual processes and regions involved in initial syntactic processes.

Neural language networks at birth.

The ability to learn language is a human trait. In adults and children, brain imaging studies have shown that auditory language activates a bilateral frontotemporal network with a left hemispheric dominance. It is an open question whether these activations represent the complete neural basis for language present at birth. Here we demonstrate that in 2-d-old infants, the language-related neural substrate is fully active in both hemispheres with a preponderance in the right auditory cortex. Functional and structural connectivities within this neural network, however, are immature, with strong connectivities only between the two hemispheres, contrasting with the adult pattern of prevalent intrahemispheric connectivities. Thus, although the brain responds to spoken language already at birth, thereby providing a strong biological basis to acquire language, progressive maturation of intrahemispheric functional connectivity is yet to be established with language exposure as the brain develops.

Intonational phrase structure processing at different stages of syntax acquisition: ERP studies in 2-, 3-, and 6-year-old children.

This study explored the electrophysiology underlying intonational phrase processing at different stages of syntax acquisition. Developmental studies suggest that children's syntactic skills advance significantly between 2 and 3 years of age. Here, children of three age groups were tested on phrase-level prosodic processing before and after this developmental phase, while their brain activity was recorded. The Closure Positive Shift (CPS), which indexes the perception of intonational phrasing in adults, served as dependent variable. The event-related brain potentials of 3- and 6-year-olds, but not of 21-month-olds, showed a CPS. These results suggest that prosodic phrase processing, as indicated by the CPS, is established only later during children's development, pointing to a close interaction of prosody and syntax acquisition.