Improving audio-visual temporal perception through training enhances beta-band activity.

Multisensory integration strongly depends on the temporal proximity between two inputs. In the audio-visual domain, stimulus pairs with delays up to a few hundred milliseconds can be perceived as simultaneous and integrated into a unified percept. Previous research has shown that the size of this temporal window of integration can be narrowed by feedback-guided training on an audio-visual simultaneity judgment task. Yet, it has remained uncertain how the neural network that processes audio-visual asynchronies is affected by the training. In the present study, participants were trained on a 2-interval forced choice audio-visual simultaneity judgment task. We recorded their neural activity with magnetoencephalography in response to three different stimulus onset asynchronies (0 ms, each participant's individual binding window, 300 ms) before, and one day following training. The Individual Window stimulus onset asynchrony condition was derived by assessing each participant's point of subjective simultaneity. Training improved performance in both asynchronous stimulus onset conditions (300 ms, Individual Window). Furthermore, beta-band amplitude (12-30 Hz) increased from pre-compared to post-training sessions. This increase moved across central, parietal, and temporal sensors during the time window of 80-410 ms post-stimulus onset. Considering the putative role of beta oscillations in carrying feedback from higher to lower cortical areas, these findings suggest that enhanced top-down modulation of sensory processing is responsible for the improved temporal acuity after training. As beta oscillations can be assumed to also preferentially support neural communication over longer conduction delays, the widespread topography of our effect could indicate that training modulates not only processing within primary sensory cortex, but rather the communication within a large-scale network.

Significance of Beta-Band Oscillations in Autism Spectrum Disorders During Motor Response Inhibition Tasks: A MEG Study.

In Autism Spectrum Disorders (ASD), impaired response inhibition and lack of adaptation are hypothesized to underlie core ASD symptoms, such as social communication and repetitive, stereotyped behavior. Thus, the aim of the present study was to compare neural correlates of inhibition, post-error adaptation, and reaction time variability in ASD and neuro-typical control (NTC) participants by investigating possible differences in error-related changes of oscillatory MEG activity. Twelve male NTC (mean age 20.3 ± 3.7) and fourteen male patients with ASD (mean age 17.8 ± 2.9) were included in the analysis. Subjects with ASD showed increased error-related reaction time variability. MEG analysis revealed decreased beta power in the ASD group in comparison to the NTC group over the centro-parietal channels in both, the pre-stimulus and post-response interval. In the ASD group, mean centro-parietal beta power negatively correlated with dimensional autism symptoms. In both groups, false alarms were followed by an early increase in temporo-frontal theta to alpha power; and by a later decrease in alpha to beta power at central and posterior sensors. Single trial correlations were additionally studied in the ASD group, who showed a positive correlation of pre-stimulus beta power with post-response theta, alpha, and beta power, particularly after hit trials. On a broader scale, the results deliver important insights into top-down control deficits that may relate to core symptoms observed in ASD.

Predictable information in neural signals during resting state is reduced in autism spectrum disorder.

The neurophysiological underpinnings of the nonsocial symptoms of autism spectrum disorder (ASD) which include sensory and perceptual atypicalities remain poorly understood. Well-known accounts of less dominant top-down influences and more dominant bottom-up processes compete to explain these characteristics. These accounts have been recently embedded in the popular framework of predictive coding theory. To differentiate between competing accounts, we studied altered information dynamics in ASD by quantifying predictable information in neural signals. Predictable information in neural signals measures the amount of stored information that is used for the next time step of a neural process. Thus, predictable information limits the (prior) information which might be available for other brain areas, for example, to build predictions for upcoming sensory information. We studied predictable information in neural signals based on resting-state magnetoencephalography (MEG) recordings of 19 ASD patients and 19 neurotypical controls aged between 14 and 27 years. Using whole-brain beamformer source analysis, we found reduced predictable information in ASD patients across the whole brain, but in particular in posterior regions of the default mode network. In these regions, epoch-by-epoch predictable information was positively correlated with source power in the alpha and beta frequency range as well as autocorrelation decay time. Predictable information in precuneus and cerebellum was negatively associated with nonsocial symptom severity, indicating a relevance of the analysis of predictable information for clinical research in ASD. Our findings are compatible with the assumption that use or precision of prior knowledge is reduced in ASD patients.

Explaining autism spectrum disorders: central coherence vs. predictive coding theories.

In this article, we review a recent paper by Stevenson et al. (J Neurosci 34: 691-697, 2014). This paper illustrates the need to present different forms of stimuli in order to characterize the perceptual abilities of people with autism spectrum disorder (ASD). Furthermore, we will discuss their behavioral results and offer an opposing viewpoint to the suggested neuronal drivers of ASD.

Smoking experience modulates the cortical integration of vision and haptics.

Human neuroplasticity of multisensory integration has been studied mainly in the context of natural or artificial training situations in healthy subjects. However, regular smokers also offer the opportunity to assess the impact of intensive daily multisensory interactions with smoking-related objects on the neural correlates of crossmodal object processing. The present functional magnetic resonance imaging study revealed that smokers show a comparable visuo-haptic integration pattern for both smoking paraphernalia and control objects in the left lateral occipital complex, a region playing a crucial role in crossmodal object recognition. Moreover, the degree of nicotine dependence correlated positively with the magnitude of visuo-haptic integration in the left lateral occipital complex (LOC) for smoking-associated but not for control objects. In contrast, in the left LOC non-smokers displayed a visuo-haptic integration pattern for control objects, but not for smoking paraphernalia. This suggests that prolonged smoking-related multisensory experiences in smokers facilitate the merging of visual and haptic inputs in the lateral occipital complex for the respective stimuli. Studying clinical populations who engage in compulsive activities may represent an ecologically valid approach to investigating the neuroplasticity of multisensory integration.

Bilateral visual field maps in a patient with only one hemisphere.

In mammals smooth retinotopic maps of the visual field are formed along the visual processing pathway whereby the left visual field is represented in the right hemisphere and vice versa. The reorganization of retinotopic maps in the lateral geniculate nucleus (LGN) of the thalamus and early visual areas (V1-V3) is studied in a patient who was born with only one cerebral hemisphere. Before the seventh week of embryonic gestation, the development of the patient's right cerebral hemisphere terminated. Despite the complete loss of her right hemisphere (di- and telencephalon) at birth, the patient's remaining hemisphere has not only developed maps of the contralateral (right) visual hemifield but, surprisingly, also maps of the ipsilateral (left) visual hemifield. Retinal ganglion-cells changed their predetermined crossing pattern in the optic chiasm and grew to the ipsilateral LGN. In the visual cortex, islands of ipsilateral visual field representations were located along the representations of the vertical meridian. In V1, smooth and continuous maps from contra- and ipsilateral hemifield overlap each other, whereas in ventral V2 and V3 ipsilateral quarter field representations invaded small distinct cortical patches. This reveals a surprising flexibility of the self-organizing developmental mechanisms responsible for map formation.

Visual Search in Naturalistic Scenes Reveals Impaired Cognitive Processing Speed in Multiple Sclerosis.

BACKGROUND: Standardized neuropsychological testing serves to quantify cognitive impairment in multiple sclerosis (MS) patients. However, the exact mechanism underlying the translation of cognitive dysfunction into difficulties in everyday tasks has remained unclear. To answer this question, we tested if MS patients with intact vs. impaired information processing speed measured by the Symbol Digit Modalities Test (SDMT) differ in their visual search behavior during ecologically valid tasks reflecting everyday activities. METHODS: Forty-three patients with relapsing-remitting MS enrolled in an eye-tracking experiment consisting of a visual search task with naturalistic images. Patients were grouped into "impaired" and "unimpaired" according to their SDMT performance. Reaction time, accuracy and eye-tracking parameters were measured. RESULTS: The groups did not differ regarding age, gender, and visual acuity. Patients with impaired SDMT (cut-off SDMT-z-score < -1.5) performance needed more time to find and fixate the target (q = 0.006). They spent less time fixating the target (q = 0.042). Impaired patients had slower reaction times and were less accurate (both q = 0.0495) even after controlling for patients' upper extremity function. Exploratory analysis revealed that unimpaired patients had higher accuracy than impaired patients particularly when the announced target was in unexpected location (p = 0.037). Correlational analysis suggested that SDMT performance is inversely linked to the time to first fixation of the target only if the announced target was in its expected location (r = -0.498, p = 0.003 vs. r = -0.212, p = 0.229). CONCLUSION: Dysfunctional visual search behavior may be one of the mechanisms translating cognitive deficits into difficulties in everyday tasks in MS patients. Our results suggest that cognitively impaired patients search their visual environment less efficiently and this is particularly evident when top-down processes have to be employed.

Audiovisual functional magnetic resonance imaging adaptation reveals multisensory integration effects in object-related sensory cortices.

Information integration across different sensory modalities contributes to object recognition, the generation of associations and long-term memory representations. Here, we used functional magnetic resonance imaging adaptation to investigate the presence of sensory integrative effects at cortical levels as early as nonprimary auditory and extrastriate visual cortices, which are implicated in intermediate stages of object processing. Stimulation consisted of an adapting audiovisual stimulus S(1) and a subsequent stimulus S(2) from the same basic-level category (e.g., cat). The stimuli were carefully balanced with respect to stimulus complexity and semantic congruency and presented in four experimental conditions: (1) the same image and vocalization for S(1) and S(2), (2) the same image and a different vocalization, (3) different images and the same vocalization, or (4) different images and vocalizations. This two-by-two factorial design allowed us to assess the contributions of auditory and visual stimulus repetitions and changes in a statistically orthogonal manner. Responses in visual regions of right fusiform gyrus and right lateral occipital cortex were reduced for repeated visual stimuli (repetition suppression). Surprisingly, left lateral occipital cortex showed stronger responses to repeated auditory stimuli (repetition enhancement). Similarly, auditory regions of interest of the right middle superior temporal gyrus and sulcus exhibited repetition suppression to auditory repetitions and repetition enhancement to visual repetitions. Our findings of crossmodal repetition-related effects in cortices of the respective other sensory modality add to the emerging view that in human subjects sensory integrative mechanisms operate on earlier cortical processing levels than previously assumed.

Involvement of action-related brain regions in nicotine addiction.

The study of Wagner et al. (J Neurosci 31: 894-898, 2011) reveals the neural correlates of spontaneously activated action representations in smokers when subjects watch movie characters smoke. We stress the importance of differentiating how these representations are activated: while the anterior intraparietal sulcus and inferior frontal gyrus are part of the mirror neuron system of smokers, the middle frontal gyrus, premotor cortex, and superior parietal lobule represent the smoking-related tool use skills and action knowledge activated by smoking paraphernalia.

Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools.

Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis.

Predictive Coding Over the Lifespan: Increased Reliance on Perceptual Priors in Older Adults-A Magnetoencephalography and Dynamic Causal Modeling Study.

Aging is accompanied by unisensory decline. To compensate for this, two complementary strategies are potentially relied upon increasingly: first, older adults integrate more information from different sensory organs. Second, according to the predictive coding (PC) model, we form "templates" (internal models or "priors") of the environment through our experiences. It is through increased life experience that older adults may rely more on these templates compared to younger adults. Multisensory integration and predictive coding would be effective strategies for the perception of near-threshold stimuli, which may however come at the cost of integrating irrelevant information. Both strategies can be studied in multisensory illusions because these require the integration of different sensory information, as well as an internal model of the world that can take precedence over sensory input. Here, we elicited a classic multisensory illusion, the sound-induced flash illusion, in younger (mean: 27 years, N = 25) and older (mean: 67 years, N = 28) adult participants while recording the magnetoencephalogram. Older adults perceived more illusions than younger adults. Older adults had increased pre-stimulus beta-band activity compared to younger adults as predicted by microcircuit theories of predictive coding, which suggest priors and predictions are linked to beta-band activity. Transfer entropy analysis and dynamic causal modeling of pre-stimulus magnetoencephalography data revealed a stronger illusion-related modulation of cross-modal connectivity from auditory to visual cortices in older compared to younger adults. We interpret this as the neural correlate of increased reliance on a cross-modal predictive template in older adults leading to the illusory percept.

Brain regions related to tool use and action knowledge reflect nicotine dependence.

In addition to reward- and craving-related processes, habitual mechanisms play an important role in addiction. While the dorsal striatum has been proposed to code for the motivational state of habitual drug-seeking actions, the neural underpinnings of the corresponding drug-taking skills and action knowledge remain poorly understood. We used functional magnetic resonance imaging (fMRI) and a behavioral orientation affordance paradigm to investigate the neural and behavioral correlates of automatized drug-taking actions in nicotine dependence. Smokers exhibited higher fMRI activations than nonsmokers when viewing smoking-related but not when viewing control images. These group differences in fMRI activations were located not only in brain regions associated with craving and habitual learning (left ventral and dorsal striatum, dorsolateral prefrontal cortex, insula, uncus, medial frontal gyrus, right subcallosal gyrus, and bilateral parahippocampal gyrus), but also in a network of brain regions which has been strongly implicated in the encoding of action knowledge and tool use skills (bilateral premotor cortex, left superior parietal lobule, and right lateral cerebellum). A behavioral affordance reaction-time task indicated that smokers, but not nonsmokers, showed an automatized responsiveness to smoking paraphernalia similar to everyday objects. Moreover, smokers showed strong intercorrelations between fMRI activations in tool use-related brain regions, behavioral responsiveness to smoking-related cues, and severity of nicotine dependence. Apparently smoking-related action representations in smokers are stored in brain regions typically representing tool use skills and action knowledge. Most importantly, cortical and behavioral correlates of the respective drug-taking skills vary with the individual degree of nicotine dependence.

Visuohaptic convergence in a corticocerebellar network.

The processing of visual and haptic inputs, occurring either separately or jointly, is crucial for everyday-life object recognition, and has been a focus of recent neuroimaging research. Previously, visuohaptic convergence has been mostly investigated with matching-task paradigms. However, much less is known about visuohaptic convergence in the absence of additional task demands. We conducted two functional magnetic resonance imaging experiments in which subjects actively touched and/or viewed unfamiliar object stimuli without any additional task demands. In addition, we performed two control experiments with audiovisual and audiohaptic stimulation to examine the specificity of the observed visuohaptic convergence effects. We found robust visuohaptic convergence in bilateral lateral occipital cortex and anterior cerebellum. In contrast, neither the anterior cerebellum nor the lateral occipital cortex showed any involvement in audiovisual or audiohaptic convergence, indicating that multisensory convergence in these regions is specifically geared to visual and haptic inputs. These data suggest that in humans the lateral occipital cortex and the anterior cerebellum play an important role in visuohaptic processing even in the absence of additional task demands.

Cortical plasticity of audio-visual object representations.

Several regions in human temporal and frontal cortex are known to integrate visual and auditory object features. The processing of audio-visual (AV) associations in these regions has been found to be modulated by object familiarity. The aim of the present study was to explore training-induced plasticity in human cortical AV integration. We used functional magnetic resonance imaging to analyze the neural correlates of AV integration for unfamiliar artificial object sounds and images in naïve subjects (PRE training) and after a behavioral training session in which subjects acquired associations between some of these sounds and images (POST-training). In the PRE-training session, unfamiliar artificial object sounds and images were mainly integrated in right inferior frontal cortex (IFC). The POST-training results showed extended integration-related IFC activations bilaterally, and a recruitment of additional regions in bilateral superior temporal gyrus/sulcus and intraparietal sulcus. Furthermore, training-induced differential response patterns to mismatching compared with matching (i.e., associated) artificial AV stimuli were most pronounced in left IFC. These effects were accompanied by complementary training-induced congruency effects in right posterior middle temporal gyrus and fusiform gyrus. Together, these findings demonstrate that short-term cross-modal association learning was sufficient to induce plastic changes of both AV integration of object stimuli and mechanisms of AV congruency processing.

Temporal characteristics of audiovisual information processing.

In complex natural environments, auditory and visual information often have to be processed simultaneously. Previous functional magnetic resonance imaging (fMRI) studies focused on the spatial localization of brain areas involved in audiovisual (AV) information processing, but the temporal characteristics of AV information flow in these regions remained unclear. In this study, we used fMRI and a novel information-theoretic approach to study the flow of AV sensory information. Subjects passively perceived sounds and images of objects presented either alone or simultaneously. Applying the measure of mutual information, we computed for each voxel the latency in which the blood oxygenation level-dependent signal had the highest information content about the preceding stimulus. The results indicate that, after AV stimulation, the earliest informative activity occurs in right Heschl's gyrus, left primary visual cortex, and the posterior portion of the superior temporal gyrus, which is known as a region involved in object-related AV integration. Informative activity in the anterior portion of superior temporal gyrus, middle temporal gyrus, right occipital cortex, and inferior frontal cortex was found at a later latency. Moreover, AV presentation resulted in shorter latencies in multiple cortical areas compared with isolated auditory or visual presentation. The results provide evidence for bottom-up processing from primary sensory areas into higher association areas during AV integration in humans and suggest that AV presentation shortens processing time in early sensory cortices.

Cognitive Impairment in Multiple Sclerosis Is Reflected by Increased Susceptibility to the Sound-Induced Flash Illusion.

Objective: To determine whether the performance of multiple sclerosis (MS) patients in the sound-induced flash illusion (SiFi), a multisensory perceptual illusion, would reflect their cognitive impairment. Methods: We performed the SiFi task as well as an extensive neuropsychological testing in 95 subjects [39 patients with relapse-remitting MS (RRMS), 16 subjects with progressive multiple sclerosis (PMS) and 40 healthy control subjects (HC)]. Results: MS patients reported more frequently the multisensory SiFi than HC. In contrast, there were no group differences in the control conditions. Essentially, patients with progressive type of MS continued to perceive the illusion at stimulus onset asynchronies (SOA) that were more than three times longer than the SOA at which the illusion was already disrupted for healthy controls. Furthermore, MS patients' degree of cognitive impairment measured with a broad neuropsychological battery encompassing tests for memory, attention, executive functions, and fluency was predicted by their performance in the SiFi task for the longest SOA of 500 ms. Conclusions: These findings support the notion that MS patients exhibit an altered multisensory perception in the SiFi task and that their susceptibility to the perceptual illusion is negatively correlated with their neuropsychological test performance. Since MS lesions affect white matter tracts and cortical regions which seem to be involved in the transfer and processing of both crossmodal and cognitive information, this might be one possible explanation for our findings. SiFi might be considered as a brief, non-expensive, language- and education-independent screening test for cognitive deficits in MS patients.

Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas.

The cortical integration of auditory and visual features is crucial for efficient object recognition. Previous studies have shown that audiovisual (AV) integration is affected by where and when auditory and visual features occur. However, because relatively little is known about the impact of what is integrated, we here investigated the impact of semantic congruency and object familiarity on the neural correlates of AV integration. We used functional magnetic resonance imaging to identify regions involved in the integration of both (congruent and incongruent) familiar animal sounds and images and of arbitrary combinations of unfamiliar artificial sounds and object images. Unfamiliar object images and sounds were integrated in the inferior frontal cortex (IFC), possibly reflecting learning of novel AV associations. Integration of familiar, but semantically incongruent combinations also correlated with IFC activation and additionally involved the posterior superior temporal sulcus (pSTS). For highly familiar semantically congruent AV pairings, we again found AV integration effects in pSTS and additionally in superior temporal gyrus. These findings demonstrate that the neural correlates of object-related AV integration reflect both semantic congruency and familiarity of the integrated sounds and images.

Probing category selectivity for environmental sounds in the human auditory brain.

Earlier studies reported evidence suggesting distinct category-related auditory representations for environmental sounds such as animal vocalizations and tool sounds in superior and middle temporal regions of the temporal lobe. However, the degree of selectivity of these representations remains to be determined. The present study combined functional magnetic resonance imaging (fMRI) adaptation with a silent acquisition protocol to further investigate category-related auditory processing of environmental sounds. To this end, we consecutively presented pairs of sounds taken from the categories 'tool sounds' or 'animal vocalizations' with either the same or different identity/category. We examined the degree of selectivity as evidenced by adaptation effects to both or only one sound category in the course of whole-brain as well as functionally and anatomically constrained region of interest analyses. While most regions predominately in the temporal cortex showed an adaptation to both sound categories, particularly the left superior temporal gyrus (STG) and the left posterior middle temporal gyrus (pMTG) selectively adapted to animal vocalizations and tool sounds, respectively. However, the activation profiles of these regions differed with respect to the general responsiveness to sounds. While tool sounds still produced fMRI signals significantly different from fixation baseline in the STG, this was not the case for animal vocalizations in pMTG. Consistent with the interpretation of STG as an intermediate auditory processing stage, this region might differentiate auditory stimuli into categories based on variations of physical stimulus properties. However, processing in left pMTG seems to be even more restricted to action-related sounds of man-made objects.

Neurocognitive Development of the Resolution of Selective Visuo-Spatial Attention: Functional MRI Evidence From Object Tracking.

Our ability to select relevant information from the environment is limited by the resolution of attention - i.e., the minimum size of the region that can be selected. Neural mechanisms that underlie this limit and its development are not yet understood. Functional magnetic resonance imaging (fMRI) was performed during an object tracking task in 7- and 11-year-old children, and in young adults. Object tracking activated canonical fronto-parietal attention systems and motion-sensitive area MT in children as young as 7 years. Object tracking performance improved with age, together with stronger recruitment of parietal attention areas and a shift from low-level to higher-level visual areas. Increasing the required resolution of spatial attention - which was implemented by varying the distance between target and distractors in the object tracking task - led to activation increases in fronto-insular cortex, medial frontal cortex including anterior cingulate cortex (ACC) and supplementary motor area, superior colliculi, and thalamus. This core circuitry for attentional precision was recruited by all age groups, but ACC showed an age-related activation reduction. Our results suggest that age-related improvements in selective visual attention and in the resolution of attention are characterized by an increased use of more functionally specialized brain regions during the course of development.

Automatized smoking-related action schemata are reflected by reduced fMRI activity in sensorimotor brain regions of smokers.

In the later stages of addiction, automatized processes play a prominent role in guiding drug-seeking and drug-taking behavior. However, little is known about the neural correlates of automatized drug-taking skills and drug-related action knowledge in humans. We employed functional magnetic resonance imaging (fMRI) while smokers and non-smokers performed an orientation affordance task, where compatibility between the hand used for a behavioral response and the spatial orientation of a priming stimulus leads to shorter reaction times resulting from activation of the corresponding motor representations. While non-smokers exhibited this behavioral effect only for control objects, smokers showed the affordance effect for both control and smoking-related objects. Furthermore, smokers exhibited reduced fMRI activation for smoking-related as compared to control objects for compatible stimulus-response pairings in a sensorimotor brain network consisting of the right primary motor cortex, supplementary motor area, middle occipital gyrus, left fusiform gyrus and bilateral cingulate gyrus. In the incompatible condition, we found higher fMRI activation in smokers for smoking-related as compared to control objects in the right primary motor cortex, cingulate gyrus, and left fusiform gyrus. This suggests that the activation and performance of deeply embedded, automatized drug-taking schemata employ less brain resources. This might reduce the threshold for relapsing in individuals trying to abstain from smoking. In contrast, the interruption or modification of already triggered automatized action representations require increased neural resources.