Evaluating the effect of denoising submillimeter auditory fMRI data with NORDIC.

Functional magnetic resonance imaging (fMRI) has emerged as an essential tool for exploring human brain function. Submillimeter fMRI, in particular, has emerged as a tool to study mesoscopic computations. The inherently low signal-to-noise ratio (SNR) at submillimeter resolutions warrants the use of denoising approaches tailored at reducing thermal noise - the dominant contributing noise component in high resolution fMRI. NORDIC PCA is one of such approaches, and has been benchmarked against other approaches in several applications. Here, we investigate the effects that two versions of NORDIC denoising have on auditory submillimeter data. As investigating auditory functional responses poses unique challenges, we anticipated that the benefit of this technique would be especially pronounced. Our results show that NORDIC denoising improves the detection sensitivity and the reliability of estimates in submillimeter auditory fMRI data. These effects can be explained by the reduction of the noise-induced signal variability. However, we also observed a reduction in the average response amplitude (percent signal), which may suggest that a small amount of signal was also removed. We conclude that, while evaluating the effects of the signal reduction induced by NORDIC may be necessary for each application, using NORDIC in high resolution auditory fMRI studies may be advantageous because of the large reduction in variability of the estimated responses.

It is a matter of perspective: Attentional focus rather than dietary restraint drives brain responses to food stimuli.

Brain responses to food are thought to reflect food's rewarding value and to fluctuate with dietary restraint. We propose that brain responses to food are dynamic and depend on attentional focus. Food pictures (high-caloric/low-caloric, palatable/unpalatable) were presented during fMRI-scanning, while attentional focus (hedonic/health/neutral) was induced in 52 female participants varying in dietary restraint. The level of brain activity was hardly different between palatable versus unpalatable foods or high-caloric versus low-caloric foods. Activity in several brain regions was higher in hedonic than in health or neutral attentional focus (p < .05, FWE-corrected). Palatability and calorie content could be decoded from multi-voxel activity patterns (p < .05, FDR-corrected). Dietary restraint did not significantly influence brain responses to food. So, level of brain activity in response to food stimuli depends on attentional focus, and may reflect salience, not reward value. Palatability and calorie content are reflected in patterns of brain activity.

Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds.

Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7 T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl's gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl's gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.

More complex than you might think: Neural representations of food reward value in obesity.

Obesity reached pandemic proportions and weight-loss treatments are mostly ineffective. The level of brain activity in the reward circuitry is proposed to be proportionate to the reward value of food stimuli, and stronger in people with obesity. However, empirical evidence is inconsistent. This may be due to the double-sided nature of high caloric palatable foods: at once highly palatable and high in calories (unhealthy). This study hypothesizes that, viewing high caloric palatable foods, a hedonic attentional focus compared to a health and a neutral attentional focus elicits more activity in reward-related brain regions, mostly in people with obesity. Moreover, caloric content and food palatability can be decoded from multivoxel patterns of activity most accurately in people with obesity and in the corresponding attentional focus. During one fMRI-session, attentional focus (hedonic, health, neutral) was manipulated using a one-back task with individually tailored food stimuli in 32 healthy-weight people and 29 people with obesity. Univariate analyses (p < 0.05, FWE-corrected) showed that brain activity was not different for palatable vs. unpalatable foods, nor for high vs. low caloric foods. Instead, this was higher in the hedonic compared to the health and neutral attentional focus. Multivariate analyses (MVPA) (p < 0.05, FDR-corrected) showed that palatability and caloric content could be decoded above chance level, independently of either BMI or attentional focus. Thus, brain activity to visual food stimuli is neither proportionate to the reward value (palatability and/or caloric content), nor significantly moderated by BMI. Instead, it depends on people's attentional focus, and may reflect motivational salience. Furthermore, food palatability and caloric content are represented as patterns of brain activity, independently of BMI and attentional focus. So, food reward value is reflected in patterns, not levels, of brain activity.

Modulating Cortical Instrument Representations During Auditory Stream Segregation and Integration With Polyphonic Music.

Numerous neuroimaging studies demonstrated that the auditory cortex tracks ongoing speech and that, in multi-speaker environments, tracking of the attended speaker is enhanced compared to the other irrelevant speakers. In contrast to speech, multi-instrument music can be appreciated by attending not only on its individual entities (i.e., segregation) but also on multiple instruments simultaneously (i.e., integration). We investigated the neural correlates of these two modes of music listening using electroencephalography (EEG) and sound envelope tracking. To this end, we presented uniquely composed music pieces played by two instruments, a bassoon and a cello, in combination with a previously validated music auditory scene analysis behavioral paradigm (Disbergen et al., 2018). Similar to results obtained through selective listening tasks for speech, relevant instruments could be reconstructed better than irrelevant ones during the segregation task. A delay-specific analysis showed higher reconstruction for the relevant instrument during a middle-latency window for both the bassoon and cello and during a late window for the bassoon. During the integration task, we did not observe significant attentional modulation when reconstructing the overall music envelope. Subsequent analyses indicated that this null result might be due to the heterogeneous strategies listeners employ during the integration task. Overall, our results suggest that subsequent to a common processing stage, top-down modulations consistently enhance the relevant instrument's representation during an instrument segregation task, whereas such an enhancement is not observed during an instrument integration task. These findings extend previous results from speech tracking to the tracking of multi-instrument music and, furthermore, inform current theories on polyphonic music perception.

Cross-validation and permutations in MVPA: Validity of permutation strategies and power of cross-validation schemes.

Multi-Voxel Pattern Analysis (MVPA) is a well established tool to disclose weak, distributed effects in brain activity patterns. The generalization ability is assessed by testing the learning model on new, unseen data. However, when limited data is available, the decoding success is estimated using cross-validation. There is general consensus on assessing statistical significance of cross-validated accuracy with non-parametric permutation tests. In this work we focus on the false positive control of different permutation strategies and on the statistical power of different cross-validation schemes. With simulations, we show that estimating the entire cross-validation error on each permuted dataset is the only statistically valid permutation strategy. Furthermore, using both simulations and real data from the HCP WU-Minn 3T fMRI dataset, we show that, among the different cross-validation schemes, a repeated split-half cross-validation is the most powerful, despite achieving slightly lower classification accuracy, when compared to other schemes. Our findings provide additional insights into the optimization of the experimental design for MVPA, highlighting the benefits of having many short runs.

Investigating the Reliability of Population Receptive Field Size Estimates Using fMRI.

In functional MRI (fMRI), population receptive field (pRF) models allow a quantitative description of the response as a function of the features of the stimuli that are relevant for each voxel. The most popular pRF model used in fMRI assumes a Gaussian shape in the features space (e.g., the visual field) reducing the description of the voxel's pRF to the Gaussian mean (the pRF preferred feature) and standard deviation (the pRF size). The estimation of the pRF mean has been proven to be highly reliable. However, the estimate of the pRF size has been shown not to be consistent within and between subjects. While this issue has been noted experimentally, here we use an optimization theory perspective to describe how the inconsistency in estimating the pRF size is linked to an inherent property of the Gaussian pRF model. When fitting such models, the goodness of fit is less sensitive to variations in the pRF size than to variations in the pRF mean. We also show how the same issue can be considered from a bias-variance perspective. We compare different estimation procedures in terms of the reliability of their estimates using simulated and real fMRI data in the visual (using the Human Connectome Project database) and auditory domain. We show that, the reliability of the estimate of the pRF size can be improved considering a linear combination of those pRF models with similar goodness of fit or a permutation based approach. This increase in reliability of the pRF size estimate does not affect the reliability of the estimate of the pRF mean and the prediction accuracy.

Behavioral effects of rhythm, carrier frequency and temporal cueing on the perception of sound sequences.

Regularity of acoustic rhythms allows predicting a target embedded within a stream thereby improving detection performance and reaction times in spectral detection tasks. In two experiments we examine whether temporal regularity enhances perceptual sensitivity and reduces reaction times using a temporal shift detection task. Participants detected temporal shifts embedded at different positions within a sequence of quintet-sounds. Narrowband quintets were centered around carrier frequencies of 200 Hz, 1100 Hz, or 3100 Hz and presented at presentation rates between 1-8 Hz. We compared rhythmic sequences to control conditions where periodicity was reduced or absent and tested whether perceptual benefits depend on the presentation rate, the spectral content of the sounds, and task difficulty. We found that (1) the slowest rate (1 Hz) led to the largest behavioral effect on sensitivity. (2) This sensitivity improvement is carrier-dependent, such that the largest improvement is observed for low-frequency (200 Hz) carriers compared to 1100 Hz and 3100 Hz carriers. (3) Moreover, we show that the predictive value of a temporal cue and that of a temporal rhythm similarly affect perceptual sensitivity. That is, both the cue and the rhythm induce confident temporal expectancies in contrast to an aperiodic rhythm, and thereby allow to effectively prepare and allocate attentional resources in time. (4) Lastly, periodic stimulation reduces reaction times compared to aperiodic stimulation, both at perceptual threshold as well as above threshold. Similarly, a temporal cue allowed participants to optimally prepare and thereby respond fastest. Overall, our results are consistent with the hypothesis that periodicity leads to optimized predictions and processing of forthcoming input and thus to behavioral benefits. Predictable temporally cued sounds provide a similar perceptual benefit to periodic rhythms, despite an additional uncertainty of target position within periodic sequences. Several neural mechanisms may underlie our findings, including the entrainment of oscillatory activity of neural populations.

Combining Gamma With Alpha and Beta Power Modulation for Enhanced Cortical Mapping in Patients With Focal Epilepsy.

About one third of patients with epilepsy have seizures refractory to the medical treatment. Electrical stimulation mapping (ESM) is the gold standard for the identification of "eloquent" areas prior to resection of epileptogenic tissue. However, it is time-consuming and may cause undesired side effects. Broadband gamma activity (55-200 Hz) recorded with extraoperative electrocorticography (ECoG) during cognitive tasks may be an alternative to ESM but until now has not proven of definitive clinical value. Considering their role in cognition, the alpha (8-12 Hz) and beta (15-25 Hz) bands could further improve the identification of eloquent cortex. We compared gamma, alpha and beta activity, and their combinations for the identification of eloquent cortical areas defined by ESM. Ten patients with intractable focal epilepsy (age: 35.9 ± 9.1 years, range: 22-48, 8 females, 9 right handed) participated in a delayed-match-to-sample task, where syllable sounds were compared to visually presented letters. We used a generalized linear model (GLM) approach to find the optimal weighting of each band for predicting ESM-defined categories and estimated the diagnostic ability by calculating the area under the receiver operating characteristic (ROC) curve. Gamma activity increased more in eloquent than in non-eloquent areas, whereas alpha and beta power decreased more in eloquent areas. Diagnostic ability of each band was close to 0.7 for all bands but depended on multiple factors including the time period of the cognitive task, the location of the electrodes and the patient's degree of attention to the stimulus. We show that diagnostic ability can be increased by 3-5% by combining gamma and alpha and by 7.5-11% when gamma and beta were combined. We then show how ECoG power modulation from cognitive testing can be used to map the probability of eloquence in individual patients and how this probability map can be used in clinical settings to optimize ESM planning. We conclude that the combination of gamma and beta power modulation during cognitive testing can contribute to the identification of eloquent areas prior to ESM in patients with refractory focal epilepsy.

Reorganization of Sound Location Processing in the Auditory Cortex of Blind Humans.

Auditory spatial tasks induce functional activation in the occipital-visual-cortex of early blind humans. Less is known about the effects of blindness on auditory spatial processing in the temporal-auditory-cortex. Here, we investigated spatial (azimuth) processing in congenitally and early blind humans with a phase-encoding functional magnetic resonance imaging (fMRI) paradigm. Our results show that functional activation in response to sounds in general-independent of sound location-was stronger in the occipital cortex but reduced in the medial temporal cortex of blind participants in comparison with sighted participants. Additionally, activation patterns for binaural spatial processing were different for sighted and blind participants in planum temporale. Finally, fMRI responses in the auditory cortex of blind individuals carried less information on sound azimuth position than those in sighted individuals, as assessed with a 2-channel, opponent coding model for the cortical representation of sound azimuth. These results indicate that early visual deprivation results in reorganization of binaural spatial processing in the auditory cortex and that blind individuals may rely on alternative mechanisms for processing azimuth position.

Transfer learning of deep neural network representations for fMRI decoding.

BACKGROUND: Deep neural networks have revolutionised machine learning, with unparalleled performance in object classification. However, in brain imaging (e.g., fMRI), the direct application of Convolutional Neural Networks (CNN) to decoding subject states or perception from imaging data seems impractical given the scarcity of available data. NEW METHOD: In this work we propose a robust method to transfer information from deep learning (DL) features to brain fMRI data with the goal of decoding. By adopting Reduced Rank Regression with Ridge Regularisation we establish a multivariate link between imaging data and the fully connected layer (fc7) of a CNN. We exploit the reconstructed fc7 features by performing an object image classification task on two datasets: one of the largest fMRI databases, taken from different scanners from more than two hundred subjects watching different movie clips, and another with fMRI data taken while watching static images. RESULTS: The fc7 features could be significantly reconstructed from the imaging data, and led to significant decoding performance. COMPARISON WITH EXISTING METHODS: The decoding based on reconstructed fc7 outperformed the decoding based on imaging data alone. CONCLUSION: In this work we show how to improve fMRI-based decoding benefiting from the mapping between functional data and CNN features. The potential advantage of the proposed method is twofold: the extraction of stimuli representations by means of an automatic procedure (unsupervised) and the embedding of high-dimensional neuroimaging data onto a space designed for visual object discrimination, leading to a more manageable space from dimensionality point of view.

How to test for phasic modulation of neural and behavioural responses.

Research on whether perception or other processes depend on the phase of neural oscillations is rapidly gaining popularity. However, it is unknown which methods are optimally suited to evaluate the hypothesized phase effect. Using a simulation approach, we here test the ability of different methods to detect such an effect on dichotomous (e.g., "hit" vs "miss") and continuous (e.g., scalp potentials) response variables. We manipulated parameters that characterise the phase effect or define the experimental approach to test for this effect. For each parameter combination and response variable, we identified an optimal method. We found that methods regressing single-trial responses on circular (sine and cosine) predictors perform best for all of the simulated parameters, regardless of the nature of the response variable (dichotomous or continuous). In sum, our study lays a foundation for optimized experimental designs and analyses in future studies investigating the role of phase for neural and behavioural responses. We provide MATLAB code for the statistical methods tested.

Methods for computing the maximum performance of computational models of fMRI responses.

Computational neuroimaging methods aim to predict brain responses (measured e.g. with functional magnetic resonance imaging [fMRI]) on the basis of stimulus features obtained through computational models. The accuracy of such prediction is used as an indicator of how well the model describes the computations underlying the brain function that is being considered. However, the prediction accuracy is bounded by the proportion of the variance of the brain response which is related to the measurement noise and not to the stimuli (or cognitive functions). This bound to the performance of a computational model has been referred to as the noise ceiling. In previous fMRI applications two methods have been proposed to estimate the noise ceiling based on either a split-half procedure or Monte Carlo simulations. These methods make different assumptions over the nature of the effects underlying the data, and, importantly, their relation has not been clarified yet. Here, we derive an analytical form for the noise ceiling that does not require computationally expensive simulations or a splitting procedure that reduce the amount of data. The validity of this analytical definition is proved in simulations, we show that the analytical solution results in the same estimate of the noise ceiling as the Monte Carlo method. Considering different simulated noise structure, we evaluate different estimators of the variance of the responses and their impact on the estimation of the noise ceiling. We furthermore evaluate the interplay between regularization (often used to estimate model fits to the data when the number of computational features in the model is large) and model complexity on the performance with respect to the noise ceiling. Our results indicate that when considering the variance of the responses across runs, computing the noise ceiling analytically results in similar estimates as the split half estimator and approaches the true noise ceiling under a variety of simulated noise scenarios. Finally, the methods are tested on real fMRI data acquired at 7 Tesla.

Optimizing fMRI experimental design for MVPA-based BCI control: Combining the strengths of block and event-related designs.

Functional Magnetic Resonance Imaging (fMRI) has been successfully used for Brain Computer Interfacing (BCI) to classify (imagined) movements of different limbs. However, reliable classification of more subtle signals originating from co-localized neural networks in the sensorimotor cortex, e.g. individual movements of fingers of the same hand, has proved to be more challenging, especially when taking into account the requirement for high single trial reliability in the BCI context. In recent years, Multi Voxel Pattern Analysis (MVPA) has gained momentum as a suitable method to disclose such weak, distributed activation patterns. Much attention has been devoted to developing and validating data analysis strategies, but relatively little guidance is available on the choice of experimental design, even less so in the context of BCI-MVPA. When applicable, block designs are considered the safest choice, but the expectations, strategies and adaptation induced by blocking of similar trials can make it a sub-optimal strategy. Fast event-related designs, in contrast, require a more complicated analysis and show stronger dependence on linearity assumptions but allow for randomly alternating trials. However, they lack resting intervals that enable the BCI participant to process feedback. In this proof-of-concept paper a hybrid blocked fast-event related design is introduced that is novel in the context of MVPA and BCI experiments, and that might overcome these issues by combining the rest periods of the block design with the shorter and randomly alternating trial characteristics of a rapid event-related design. A well-established button-press experiment was used to perform a within-subject comparison of the proposed design with a block and a slow event-related design. The proposed hybrid blocked fast-event related design showed a decoding accuracy that was close to that of the block design, which showed highest accuracy. It allowed for across-design decoding, i.e. reliable prediction of examples obtained with another design. Finally, it also showed the most stable incremental decoding results, obtaining good performance with relatively few blocks. Our findings suggest that the blocked fast event-related design could be a viable alternative to block designs in the context of BCI-MVPA, when expectations, strategies and adaptation make blocking of trials of the same type a sub-optimal strategy. Additionally, the blocked fast event-related design is also suitable for applications in which fast incremental decoding is desired, and enables the use of a slow or block design during the test phase.

Topographic Somatosensory Imagery for Real-Time fMRI Brain-Computer Interfacing.

Real-time functional magnetic resonance imaging (fMRI) is a promising non-invasive method for brain-computer interfaces (BCIs). BCIs translate brain activity into signals that allow communication with the outside world. Visual and motor imagery are often used as information-encoding strategies, but can be challenging if not grounded in recent experience in these modalities, e.g., in patients with locked-in-syndrome (LIS). In contrast, somatosensory imagery might constitute a more suitable information-encoding strategy as the somatosensory function is often very robust. Somatosensory imagery has been shown to activate the somatotopic cortex, but it has been unclear so far whether it can be reliably detected on a single-trial level and successfully classified according to specific somatosensory imagery content. Using ultra-high field 7-T fMRI, we show reliable and high-accuracy single-trial decoding of left-foot (LF) vs. right-hand (RH) somatosensory imagery. Correspondingly, higher decoding accuracies were associated with greater spatial separation of hand and foot decoding-weight patterns in the primary somatosensory cortex (S1). Exploiting these novel neuroscientific insights, we developed-and provide a proof of concept for-basic BCI communication by showing that binary (yes/no) answers encoded by somatosensory imagery can be decoded with high accuracy in simulated real-time (in 7 subjects) as well as in real-time (1 subject). This study demonstrates that body part-specific somatosensory imagery differentially activates somatosensory cortex in a topographically specific manner; evidence which was surprisingly still lacking in the literature. It also offers proof of concept for a novel somatosensory imagery-based fMRI-BCI control strategy, with particularly high potential for visually and motor-impaired patients. The strategy could also be transferred to lower MRI field strengths and to mobile functional near-infrared spectroscopy. Finally, given that communication BCIs provide the BCI user with a form of feedback based on their brain signals and can thus be considered as a specific form of neurofeedback, and that repeated use of a BCI has been shown to enhance underlying representations, we expect that the current BCI could also offer an interesting new approach for somatosensory rehabilitation training in the context of stroke and phantom limb pain.

Active Sound Localization Sharpens Spatial Tuning in Human Primary Auditory Cortex.

Spatial hearing sensitivity in humans is dynamic and task-dependent, but the mechanisms in human auditory cortex that enable dynamic sound location encoding remain unclear. Using functional magnetic resonance imaging (fMRI), we assessed how active behavior affects encoding of sound location (azimuth) in primary auditory cortical areas and planum temporale (PT). According to the hierarchical model of auditory processing and cortical functional specialization, PT is implicated in sound location ("where") processing. Yet, our results show that spatial tuning profiles in primary auditory cortical areas (left primary core and right caudo-medial belt) sharpened during a sound localization ("where") task compared with a sound identification ("what") task. In contrast, spatial tuning in PT was sharp but did not vary with task performance. We further applied a population pattern decoder to the measured fMRI activity patterns, which confirmed the task-dependent effects in the left core: sound location estimates from fMRI patterns measured during active sound localization were most accurate. In PT, decoding accuracy was not modulated by task performance. These results indicate that changes of population activity in human primary auditory areas reflect dynamic and task-dependent processing of sound location. As such, our findings suggest that the hierarchical model of auditory processing may need to be revised to include an interaction between primary and functionally specialized areas depending on behavioral requirements.SIGNIFICANCE STATEMENT According to a purely hierarchical view, cortical auditory processing consists of a series of analysis stages from sensory (acoustic) processing in primary auditory cortex to specialized processing in higher-order areas. Posterior-dorsal cortical auditory areas, planum temporale (PT) in humans, are considered to be functionally specialized for spatial processing. However, this model is based mostly on passive listening studies. Our results provide compelling evidence that active behavior (sound localization) sharpens spatial selectivity in primary auditory cortex, whereas spatial tuning in functionally specialized areas (PT) is narrow but task-invariant. These findings suggest that the hierarchical view of cortical functional specialization needs to be extended: our data indicate that active behavior involves feedback projections from higher-order regions to primary auditory cortex.

Cortical tracking of multiple streams outside the focus of attention in naturalistic auditory scenes.

In everyday life, we process mixtures of a variety of sounds. This processing involves the segregation of auditory input and the attentive selection of the stream that is most relevant to current goals. For natural scenes with multiple irrelevant sounds, however, it is unclear how the human auditory system represents all the unattended sounds. In particular, it remains elusive whether the sensory input to the human auditory cortex of unattended sounds biases the cortical integration/segregation of these sounds in a similar way as for attended sounds. In this study, we tested this by asking participants to selectively listen to one of two speakers or music in an ongoing 1-min sound mixture while their cortical neural activity was measured with EEG. Using a stimulus reconstruction approach, we find better reconstruction of mixed unattended sounds compared to individual unattended sounds at two early cortical stages (70 ms and 150 ms) of the auditory processing hierarchy. Crucially, at the earlier processing stage (70 ms), this cortical bias to represent unattended sounds as integrated rather than segregated increases with increasing similarity of the unattended sounds. Our results reveal an important role of acoustical properties for the cortical segregation of unattended auditory streams in natural listening situations. They further corroborate the notion that selective attention contributes functionally to cortical stream segregation. These findings highlight that a common, acoustics-based grouping principle governs the cortical representation of auditory streams not only inside but also outside the listener's focus of attention.

Sensorimotor Representation of Speech Perception. Cross-Decoding of Place of Articulation Features during Selective Attention to Syllables in 7T fMRI.

Sensorimotor integration, the translation between acoustic signals and motoric programs, may constitute a crucial mechanism for speech. During speech perception, the acoustic-motoric translations include the recruitment of cortical areas for the representation of speech articulatory features, such as place of articulation. Selective attention can shape the processing and performance of speech perception tasks. Whether and where sensorimotor integration takes place during attentive speech perception remains to be explored. Here, we investigate articulatory feature representations of spoken consonant-vowel (CV) syllables during two distinct tasks. Fourteen healthy humans attended to either the vowel or the consonant within a syllable in separate delayed-match-to-sample tasks. Single-trial fMRI blood oxygenation level-dependent (BOLD) responses from perception periods were analyzed using multivariate pattern classification and a searchlight approach to reveal neural activation patterns sensitive to the processing of place of articulation (i.e., bilabial/labiodental vs. alveolar). To isolate place of articulation representation from acoustic covariation, we applied a cross-decoding (generalization) procedure across distinct features of manner of articulation (i.e., stop, fricative, and nasal). We found evidence for the representation of place of articulation across tasks and in both tasks separately: for attention to vowels, generalization maps included bilateral clusters of superior and posterior temporal, insular, and frontal regions; for attention to consonants, generalization maps encompassed clusters in temporoparietal, insular, and frontal regions within the right hemisphere only. Our results specify the cortical representation of place of articulation features generalized across manner of articulation during attentive syllable perception, thus supporting sensorimotor integration during attentive speech perception and demonstrating the value of generalization.

Assessing Top-Down and Bottom-Up Contributions to Auditory Stream Segregation and Integration With Polyphonic Music.

Polyphonic music listening well exemplifies processes typically involved in daily auditory scene analysis situations, relying on an interactive interplay between bottom-up and top-down processes. Most studies investigating scene analysis have used elementary auditory scenes, however real-world scene analysis is far more complex. In particular, music, contrary to most other natural auditory scenes, can be perceived by either integrating or, under attentive control, segregating sound streams, often carried by different instruments. One of the prominent bottom-up cues contributing to multi-instrument music perception is their timbre difference. In this work, we introduce and validate a novel paradigm designed to investigate, within naturalistic musical auditory scenes, attentive modulation as well as its interaction with bottom-up processes. Two psychophysical experiments are described, employing custom-composed two-voice polyphonic music pieces within a framework implementing a behavioral performance metric to validate listener instructions requiring either integration or segregation of scene elements. In Experiment 1, the listeners' locus of attention was switched between individual instruments or the aggregate (i.e., both instruments together), via a task requiring the detection of temporal modulations (i.e., triplets) incorporated within or across instruments. Subjects responded post-stimulus whether triplets were present in the to-be-attended instrument(s). Experiment 2 introduced the bottom-up manipulation by adding a three-level morphing of instrument timbre distance to the attentional framework. The task was designed to be used within neuroimaging paradigms; Experiment 2 was additionally validated behaviorally in the functional Magnetic Resonance Imaging (fMRI) environment. Experiment 1 subjects (N = 29, non-musicians) completed the task at high levels of accuracy, showing no group differences between any experimental conditions. Nineteen listeners also participated in Experiment 2, showing a main effect of instrument timbre distance, even though within attention-condition timbre-distance contrasts did not demonstrate any timbre effect. Correlation of overall scores with morph-distance effects, computed by subtracting the largest from the smallest timbre distance scores, showed an influence of general task difficulty on the timbre distance effect. Comparison of laboratory and fMRI data showed scanner noise had no adverse effect on task performance. These Experimental paradigms enable to study both bottom-up and top-down contributions to auditory stream segregation and integration within psychophysical and neuroimaging experiments.