Cortical network responses map onto data-driven features that capture visual semantics of movie fragments.

Research on how the human brain extracts meaning from sensory input relies in principle on methodological reductionism. In the present study, we adopt a more holistic approach by modeling the cortical responses to semantic information that was extracted from the visual stream of a feature film, employing artificial neural network models. Advances in both computer vision and natural language processing were utilized to extract the semantic representations from the film by combining perceptual and linguistic information. We tested whether these representations were useful in studying the human brain data. To this end, we collected electrocorticography responses to a short movie from 37 subjects and fitted their cortical patterns across multiple regions using the semantic components extracted from film frames. We found that individual semantic components reflected fundamental semantic distinctions in the visual input, such as presence or absence of people, human movement, landscape scenes, human faces, etc. Moreover, each semantic component mapped onto a distinct functional cortical network involving high-level cognitive regions in occipitotemporal, frontal and parietal cortices. The present work demonstrates the potential of the data-driven methods from information processing fields to explain patterns of cortical responses, and contributes to the overall discussion about the encoding of high-level perceptual information in the human brain.

Towards an intuitive communication-BCI: decoding visually imagined characters from the early visual cortex using high-field fMRI.

Brain-computer interfaces aim to provide people with paralysis with the possibility to use their neural signals to control devices. For communication, most BCIs are based on the selection of letters from a (digital) letter board to spell words and sentences. Visual mental imagery of letters could offer a new, fast and intuitive way to spell in a BCI-communication solution. Here we provide a proof of concept for the decoding of visually imagined characters from the early visual cortex using 7 Tesla functional MRI. Sixteen healthy participants visually imagined three different characters for 3, 5 and 7 s in a slow event-related design. Using single-trial classification, we were able to decode the characters with an average accuracy of 54%, which is significantly above chance level (33%). Furthermore, the imagined characters were classifiable shortly after cue onset and remained classifiable with prolonged imagery. These properties, combined with the cortical location of the early visual cortex and its decodable activity, encourage further research on intracranial interfacing using surface electrodes to bring us closer to such a visual imagery based BCI communication solution.

Neural Tuning to Low-Level Features of Speech throughout the Perisylvian Cortex.

Despite a large body of research, we continue to lack a detailed account of how auditory processing of continuous speech unfolds in the human brain. Previous research showed the propagation of low-level acoustic features of speech from posterior superior temporal gyrus toward anterior superior temporal gyrus in the human brain (Hullett et al., 2016). In this study, we investigate what happens to these neural representations past the superior temporal gyrus and how they engage higher-level language processing areas such as inferior frontal gyrus. We used low-level sound features to model neural responses to speech outside of the primary auditory cortex. Two complementary imaging techniques were used with human participants (both males and females): electrocorticography (ECoG) and fMRI. Both imaging techniques showed tuning of the perisylvian cortex to low-level speech features. With ECoG, we found evidence of propagation of the temporal features of speech sounds along the ventral pathway of language processing in the brain toward inferior frontal gyrus. Increasingly coarse temporal features of speech spreading from posterior superior temporal cortex toward inferior frontal gyrus were associated with linguistic features such as voice onset time, duration of the formant transitions, and phoneme, syllable, and word boundaries. The present findings provide the groundwork for a comprehensive bottom-up account of speech comprehension in the human brain.SIGNIFICANCE STATEMENT We know that, during natural speech comprehension, a broad network of perisylvian cortical regions is involved in sound and language processing. Here, we investigated the tuning to low-level sound features within these regions using neural responses to a short feature film. We also looked at whether the tuning organization along these brain regions showed any parallel to the hierarchy of language structures in continuous speech. Our results show that low-level speech features propagate throughout the perisylvian cortex and potentially contribute to the emergence of "coarse" speech representations in inferior frontal gyrus typically associated with high-level language processing. These findings add to the previous work on auditory processing and underline a distinctive role of inferior frontal gyrus in natural speech comprehension.

Neurotoxic effects of ecstasy on the thalamus.

BACKGROUND: Neurotoxic effects of ecstasy have been reported, although it remains unclear whether effects can be attributed to ecstasy, other recreational drugs or a combination of these. AIMS: To assess specific/independent neurotoxic effects of heavy ecstasy use and contributions of amphetamine, cocaine and cannabis as part of The Netherlands XTC Toxicity (NeXT) study. METHOD: Effects of ecstasy and other substances were assessed with (1)H-magnetic resonance spectroscopy, diffusion tensor imaging, perfusion weighted imaging and [(123)I]2beta-carbomethoxy-3beta-(4-iodophenyl)-tropane ([(123)I]beta-CIT) single photon emission computed tomography (serotonin transporters) in a sample (n=71) with broad variation in drug use, using multiple regression analyses. RESULTS: Ecstasy showed specific effects in the thalamus with decreased [(123)I]beta-CIT binding, suggesting serotonergic axonal damage; decreased fractional anisotropy, suggesting axonal loss; and increased cerebral blood volume probably caused by serotonin depletion. Ecstasy had no effect on brain metabolites and apparent diffusion coefficients. CONCLUSIONS: Converging evidence was found for a specific toxic effect of ecstasy on serotonergic axons in the thalamus.

Decreased thalamic blood flow in obsessive-compulsive disorder patients responding to fluvoxamine.

Functional imaging studies have pointed to a role of the orbitofrontal cortex (OFC), striatum and thalamus in the pathophysiology of obsessive-compulsive disorder (OCD). Effective treatment has been found to change brain activity within this circuitry. The aim of the present study was to explore possible differential effects of OCD responders and non-responders to drug treatment on the regional cerebral blood flow (rCBF). Measurements of rCBF were carried out in 15 out of 22 patients with OCD who completed an open-label trial with fluvoxamine. Patients were studied with 99mTc-HMPAO single photon emission computed tomography (SPECT) before and after 12 weeks of treatment. In addition, structural magnetic resonance imaging was obtained on all patients. Regions of interest comprised the OFC, caudate nucleus, putamen and thalamus. Seven patients responded to treatment. Levels of rCBF decreased significantly in the left caudate nucleus and the left and right putamen in both responders and non-responders to treatment. In responders, but not in non-responders, a significant decrease in rCBF was found in the right thalamus. Pre-treatment cerebellar and whole brain HMPAO uptake was significantly higher in responders to treatment compared with non-responders. We suggest that the thalamus plays a central role in the response to drug treatment.

Functional anatomy of top-down visuospatial processing in the human brain: evidence from rTMS.

The hypothesis was tested that visuospatial mental imagery relies on processing in the posterior parietal lobe. Using repetitive transcranial magnetic stimulation (rTMS) in a cross-over, sham-controlled design, we compared involvement of right posterior parietal cortex with primary visual cortex. Subjects received rTMS over the parietal and occipital cortices during 20 min, after which they performed a behaviorally controlled visuospatial mental imagery task. Performance deteriorated significantly after rTMS over the parietal, but not occipital, cortex. These data support a causal link between parietal activation and top-down spatial processing.