Space as a scaffold for rotational generalisation of abstract concepts.

Learning invariances allows us to generalise. In the visual modality, invariant representations allow us to recognise objects despite translations or rotations in physical space. However, how we learn the invariances that allow us to generalise abstract patterns of sensory data ('concepts') is a longstanding puzzle. Here, we study how humans generalise relational patterns in stimulation sequences that are defined by either transitions on a nonspatial two-dimensional feature manifold, or by transitions in physical space. We measure rotational generalisation, i.e., the ability to recognise concepts even when their corresponding transition vectors are rotated. We find that humans naturally generalise to rotated exemplars when stimuli are defined in physical space, but not when they are defined as positions on a nonspatial feature manifold. However, if participants are first pre-trained to map auditory or visual features to spatial locations, then rotational generalisation becomes possible even in nonspatial domains. These results imply that space acts as a scaffold for learning more abstract conceptual invariances.

Multisensory correlation computations in the human brain identified by a time-resolved encoding model.

Neural mechanisms that arbitrate between integrating and segregating multisensory information are essential for complex scene analysis and for the resolution of the multisensory correspondence problem. However, these mechanisms and their dynamics remain largely unknown, partly because classical models of multisensory integration are static. Here, we used the Multisensory Correlation Detector, a model that provides a good explanatory power for human behavior while incorporating dynamic computations. Participants judged whether sequences of auditory and visual signals originated from the same source (causal inference) or whether one modality was leading the other (temporal order), while being recorded with magnetoencephalography. First, we confirm that the Multisensory Correlation Detector explains causal inference and temporal order behavioral judgments well. Second, we found strong fits of brain activity to the two outputs of the Multisensory Correlation Detector in temporo-parietal cortices. Finally, we report an asymmetry in the goodness of the fits, which were more reliable during the causal inference task than during the temporal order judgment task. Overall, our results suggest the existence of multisensory correlation detectors in the human brain, which explain why and how causal inference is strongly driven by the temporal correlation of multisensory signals.

Does auditory deprivation impairs statistical learning in the auditory modality?

Early sensory deprivation allows assessing the extent of reorganisation of cognitive functions, well beyond sensory processing. As such, it is a good model to explore the links between sensory experience and cognitive functions. One of these functions, statistical learning - the ability to extract and use regularities present in the environment - is suspected to be impaired in prelingually deaf children with a cochlear implant. However, empirical evidence supporting this claim is very scarce and studies have reported contradictory results. This might be because previous studies have tested statistical learning only in the visual modality and did not make clear distinctions between multiple types of statistical regularities. To overcome these problems, we designed a modified serial reaction time task where cochlear implanted children and normal hearing children had to react to auditory sequences that embed multiple statistical regularities, namely transition probabilities of 0th, 1st or 2nd order. We compared the reaction times of the children with the output of a simple computational model that learns transition probabilities. First, 6-12 years old children were able to learn 0th and 1st order transition probabilities but not 2nd order ones. Second, there were no differences between cochlear implanted children and their normal hearing peers. These results indicate that auditory statistical learning is preserved in congenitally deaf children with cochlear implants. This suggests in turn that early auditory deprivation might not be crucially detrimental for the normal development of statistical learning.

Training-induced plasticity enables visualizing sounds with a visual-to-auditory conversion device.

Sensory substitution devices aim at restoring visual functions by converting visual information into auditory or tactile stimuli. Although these devices show promise in the range of behavioral abilities they allow, the processes underlying their use remain underspecified. In particular, while an initial debate focused on the visual versus auditory or tactile nature of sensory substitution, since over a decade, the idea that it reflects a mixture of both has emerged. In order to investigate behaviorally the extent to which visual and auditory processes are involved, participants completed a Stroop-like crossmodal interference paradigm before and after being trained with a conversion device which translates visual images into sounds. In addition, participants' auditory abilities and their phenomenologies were measured. Our study revealed that, after training, when asked to identify sounds, processes shared with vision were involved, as participants' performance in sound identification was influenced by the simultaneously presented visual distractors. In addition, participants' performance during training and their associated phenomenology depended on their auditory abilities, revealing that processing finds its roots in the input sensory modality. Our results pave the way for improving the design and learning of these devices by taking into account inter-individual differences in auditory and visual perceptual strategies.

Frequency Selectivity of Persistent Cortical Oscillatory Responses to Auditory Rhythmic Stimulation.

Cortical oscillations have been proposed to play a functional role in speech and music perception, attentional selection, and working memory, via the mechanism of neural entrainment. One of the properties of neural entrainment that is often taken for granted is that its modulatory effect on ongoing oscillations outlasts rhythmic stimulation. We tested the existence of this phenomenon by studying cortical neural oscillations during and after presentation of melodic stimuli in a passive perception paradigm. Melodies were composed of ∼60 and ∼80 Hz tones embedded in a 2.5 Hz stream. Using intracranial and surface recordings in male and female humans, we reveal persistent oscillatory activity in the high-γ band in response to the tones throughout the cortex, well beyond auditory regions. By contrast, in response to the 2.5 Hz stream, no persistent activity in any frequency band was observed. We further show that our data are well captured by a model of damped harmonic oscillator and can be classified into three classes of neural dynamics, with distinct damping properties and eigenfrequencies. This model provides a mechanistic and quantitative explanation of the frequency selectivity of auditory neural entrainment in the human cortex.SIGNIFICANCE STATEMENT It has been proposed that the functional role of cortical oscillations is subtended by a mechanism of entrainment, the synchronization in phase or amplitude of neural oscillations to a periodic stimulation. One of the properties of neural entrainment that is often taken for granted is that its modulatory effect on ongoing oscillations outlasts rhythmic stimulation. Using intracranial and surface recordings of humans passively listening to rhythmic auditory stimuli, we reveal consistent oscillatory responses throughout the cortex, with persistent activity of high-γ oscillations. On the contrary, neural oscillations do not outlast low-frequency acoustic dynamics. We interpret our results as reflecting harmonic oscillator properties, a model ubiquitous in physics but rarely used in neuroscience.

Musical Expertise Is Associated with Improved Neural Statistical Learning in the Auditory Domain.

It is poorly known whether musical training is associated with improvements in general cognitive abilities, such as statistical learning (SL). In standard SL paradigms, musicians have shown better performances than nonmusicians. However, this advantage could be due to differences in auditory discrimination, in memory or truly in the ability to learn sequence statistics. Unfortunately, these different hypotheses make similar predictions in terms of expected results. To dissociate them, we developed a Bayesian model and recorded electroencephalography (EEG). Our results confirm that musicians perform approximately 15% better than nonmusicians at predicting items in auditory sequences that embed either low or high-order statistics. These higher performances are explained in the model by parameters governing the learning of high-order statistics and the selection stage noise. EEG recordings reveal a neural underpinning of the musician's advantage: the P300 amplitude correlates with the surprise elicited by each item, and so, more strongly for musicians. Finally, early EEG components correlate with the surprise elicited by low-order statistics, as opposed to late EEG components that correlate with the surprise elicited by high-order statistics and this effect is stronger for musicians. Overall, our results demonstrate that musical expertise is associated with improved neural SL in the auditory domain. SIGNIFICANCE STATEMENT: It is poorly known whether musical training leads to improvements in general cognitive skills. One fundamental cognitive ability, SL, is thought to be enhanced in musicians, but previous studies have reported mixed results. This is because such musician's advantage can embrace very different explanations, such as improvement in auditory discrimination or in memory. To solve this problem, we developed a Bayesian model and recorded EEG to dissociate these explanations. Our results reveal that musical expertise is truly associated with an improved ability to learn sequence statistics, especially high-order statistics. This advantage is reflected in the electroencephalographic recordings, where the P300 amplitude is more sensitive to surprising items in musicians than in nonmusicians.

Rhythmic Abilities of Children With Hearing Loss.

OBJECTIVES: Children with hearing loss (HL), in spite of early cochlear implantation, often struggle considerably with language acquisition. Previous research has shown a benefit of rhythmic training on linguistic skills in children with HL, suggesting that improving rhythmic capacities could help attenuating language difficulties. However, little is known about general rhythmic skills of children with HL and how they relate to speech perception. The aim of this study is twofold: (1) to assess the abilities of children with HL in different rhythmic sensorimotor synchronization tasks compared to a normal-hearing control group and (2) to investigate a possible relation between sensorimotor synchronization abilities and speech perception abilities in children with HL. DESIGN: A battery of sensorimotor synchronization tests with stimuli of varying acoustic and temporal complexity was used: a metronome, different musical excerpts, and complex rhythmic patterns. Synchronization abilities were assessed in 32 children (aged from 5 to 10 years) with a severe to profound HL mainly fitted with one or two cochlear implants (n = 28) or with hearing aids (n = 4). Working memory and sentence repetition abilities were also assessed. Performance was compared to an age-matched control group of 24 children with normal hearing. The comparison took into account variability in working memory capacities. For children with HL only, we computed linear regressions on speech, sensorimotor synchronization, and working memory abilities, including device-related variables such as onset of device use, type of device, and duration of use. RESULTS: Compared to the normal-hearing group, children with HL performed poorly in all sensorimotor synchronization tasks, but the effect size was greater for complex as compared to simple stimuli. Group differences in working memory did not explain this result. Linear regression analysis revealed that working memory, synchronization to complex rhythms performances, age, and duration of device use predicted the number of correct syllables produced in a sentence repetition task. CONCLUSION: Despite early cochlear implantation or hearing aid use, hearing impairment affects the quality of temporal processing of acoustic stimuli in congenitally deaf children. This deficit seems to be more severe with stimuli of increasing rhythmic complexity highlighting a difficulty in structuring sounds according to a temporal hierarchy.

Musical Training for Auditory Rehabilitation in Hearing Loss.

Despite the overall success of cochlear implantation, language outcomes remain suboptimal and subject to large inter-individual variability. Early auditory rehabilitation techniques have mostly focused on low-level sensory abilities. However, a new body of literature suggests that cognitive operations are critical for auditory perception remediation. We argue in this paper that musical training is a particularly appealing candidate for such therapies, as it involves highly relevant cognitive abilities, such as temporal predictions, hierarchical processing, and auditory-motor interactions. We review recent studies demonstrating that music can enhance both language perception and production at multiple levels, from syllable processing to turn-taking in natural conversation.

Rhythmic Training Improves Temporal Anticipation and Adaptation Abilities in Children With Hearing Loss During Verbal Interaction.

Purpose In this study, we investigate temporal adaptation capacities of children with normal hearing and children with cochlear implants and/or hearing aids during verbal exchange. We also address the question of the efficiency of a rhythmic training on temporal adaptation during speech interaction in children with hearing loss. Method We recorded electroencephalogram data in children while they named pictures delivered on a screen, in alternation with a virtual partner. We manipulated the virtual partner's speech rate (fast vs. slow) and the regularity of alternation (regular vs. irregular). The group of children with normal hearing was tested once, and the group of children with hearing loss was tested twice: once after 30 min of auditory training and once after 30 min of rhythmic training. Results Both groups of children adjusted their speech rate to that of the virtual partner and were sensitive to the regularity of alternation with a less accurate performance following irregular turns. Moreover, irregular turns elicited a negative event-related potential in both groups, showing a detection of temporal deviancy. Notably, the amplitude of this negative component positively correlated with accuracy in the alternation task. In children with hearing loss, the effect was more pronounced and long-lasting following rhythmic training compared with auditory training. Conclusion These results are discussed in terms of temporal adaptation abilities in speech interaction and suggest the use of rhythmic training to improve these skills of children with hearing loss.

Individual Differences in Sensory Substitution.

Sensory substitution devices were developed in the context of perceptual rehabilitation and they aim at compensating one or several functions of a deficient sensory modality by converting stimuli that are normally accessed through this deficient sensory modality into stimuli accessible by another sensory modality. For instance, they can convert visual information into sounds or tactile stimuli. In this article, we review those studies that investigated the individual differences at the behavioural, neural, and phenomenological levels when using a sensory substitution device. We highlight how taking into account individual differences has consequences for the optimization and learning of sensory substitution devices. We also discuss the extent to which these studies allow a better understanding of the experience with sensory substitution devices, and in particular how the resulting experience is not akin to a single sensory modality. Rather, it should be conceived as a multisensory experience, involving both perceptual and cognitive processes, and emerging on each user's pre-existing sensory and cognitive capacities.