Functional connectivity profiles of the default mode and visual networks reflect temporal accumulative effects of sustained naturalistic emotional experience.

Determining and decoding emotional brain processes under ecologically valid conditions remains a key challenge in affective neuroscience. The current functional Magnetic Resonance Imaging (fMRI) based emotion decoding studies are mainly based on brief and isolated episodes of emotion induction, while sustained emotional experience in naturalistic environments that mirror daily life experiences are scarce. Here we used 12 different 10-minute movie clips as ecologically valid emotion-evoking procedures in n = 52 individuals to explore emotion-specific fMRI functional connectivity (FC) profiles on the whole-brain level at high spatial resolution (432 parcellations including cortical and subcortical structures). Employing machine-learning based decoding and cross validation procedures allowed to investigate FC profiles contributing to classification that can accurately distinguish sustained happiness and sadness and that generalize across subjects, movie clips, and parcellations. Both functional brain network-based and subnetwork-based emotion classification results suggested that emotion manifests as distributed representation of multiple networks, rather than a single functional network or subnetwork. Further, the results showed that the Visual Network (VN) and Default Mode Network (DMN) associated functional networks, especially VN-DMN, exhibited a strong contribution to emotion classification. To further estimate the temporal accumulative effect of naturalistic long-term movie-based video-evoking emotions, we divided the 10-min episode into three stages: early stimulation (1∼200 s), middle stimulation (201∼400 s), and late stimulation (401∼600 s) and examined the emotion classification performance at different stimulation stages. We found that the late stimulation contributes most to the classification (accuracy=85.32%, F1-score=85.62%) compared to early and middle stimulation stages, implying that continuous exposure to emotional stimulation can lead to more intense emotions and further enhance emotion-specific distinguishable representations. The present work demonstrated that sustained happiness and sadness under naturalistic conditions are presented in emotion-specific network profiles and these expressions may play different roles in the generation and modulation of emotions. These findings elucidated the importance of network level adaptations for sustained emotional experiences during naturalistic contexts and open new venues for imaging network level contributions under naturalistic conditions.

Consistency and similarity of MEG- and fMRI-signal time courses during movie viewing.

Movie viewing allows human perception and cognition to be studied in complex, real-life-like situations in a brain-imaging laboratory. Previous studies with functional magnetic resonance imaging (fMRI) and with magneto- and electroencephalography (MEG and EEG) have demonstrated consistent temporal dynamics of brain activity across movie viewers. However, little is known about the similarities and differences of fMRI and MEG or EEG dynamics during such naturalistic situations. We thus compared MEG and fMRI responses to the same 15-min black-and-white movie in the same eight subjects who watched the movie twice during both MEG and fMRI recordings. We analyzed intra- and intersubject voxel-wise correlations within each imaging modality as well as the correlation of the MEG envelopes and fMRI signals. The fMRI signals showed voxel-wise within- and between-subjects correlations up to r = 0.66 and r = 0.37, respectively, whereas these correlations were clearly weaker for the envelopes of band-pass filtered (7 frequency bands below 100 Hz) MEG signals (within-subjects correlation r < 0.14 and between-subjects r < 0.05). Direct MEG-fMRI voxel-wise correlations were unreliable. Notably, applying a spatial-filtering approach to the MEG data uncovered consistent canonical variates that showed considerably stronger (up to r = 0.25) between-subjects correlations than the univariate voxel-wise analysis. Furthermore, the envelopes of the time courses of these variates up to about 10 Hz showed association with fMRI signals in a general linear model. Similarities between envelopes of MEG canonical variates and fMRI voxel time-courses were seen mostly in occipital, but also in temporal and frontal brain regions, whereas intra- and intersubject correlations for MEG and fMRI separately were strongest only in the occipital areas. In contrast to the conventional univariate analysis, the spatial-filtering approach was able to uncover associations between the MEG envelopes and fMRI time courses, shedding light on the similarities of hemodynamic and electromagnetic brain activities during movie viewing.

Functional brain segmentation using inter-subject correlation in fMRI.

The human brain continuously processes massive amounts of rich sensory information. To better understand such highly complex brain processes, modern neuroimaging studies are increasingly utilizing experimental setups that better mimic daily-life situations. A new exploratory data-analysis approach, functional segmentation inter-subject correlation analysis (FuSeISC), was proposed to facilitate the analysis of functional magnetic resonance (fMRI) data sets collected in these experiments. The method provides a new type of functional segmentation of brain areas, not only characterizing areas that display similar processing across subjects but also areas in which processing across subjects is highly variable. FuSeISC was tested using fMRI data sets collected during traditional block-design stimuli (37 subjects) as well as naturalistic auditory narratives (19 subjects). The method identified spatially local and/or bilaterally symmetric clusters in several cortical areas, many of which are known to be processing the types of stimuli used in the experiments. The method is not only useful for spatial exploration of large fMRI data sets obtained using naturalistic stimuli, but also has other potential applications, such as generation of a functional brain atlases including both lower- and higher-order processing areas. Finally, as a part of FuSeISC, a criterion-based sparsification of the shared nearest-neighbor graph was proposed for detecting clusters in noisy data. In the tests with synthetic data, this technique was superior to well-known clustering methods, such as Ward's method, affinity propagation, and K-means ++. Hum Brain Mapp 38:2643-2665, 2017. © 2017 Wiley Periodicals, Inc.

A correlation-based method for extracting subject-specific components and artifacts from group-fMRI data.

We present a simple but effective correlation-based method (maxCorr) for extracting subject-specific components from group-fMRI data. The method finds signal components that correlate maximally with the data set of one subject and minimally with the data sets of the other subjects. We show that such subject-specific components are often related to movement and physiological noise (e.g. cardiac cycle, respiration). We further demonstrate that removing the most subject-specific components for each subject reduces the overall data variance and improves the statistical identification of true fMRI activations. We compare the performance of maxCorr with CompCor, a commonly used artifact-finding method in fMRI analysis. We show that maxCorr is less likely than CompCor to remove actual stimulus-related activity, especially when no information about the stimulus is available. MaxCorr operates without stimulus information and is therefore well suitable for analyses of fMRI experiments employing naturalistic stimuli, such as movies, where stimulus regressors are difficult to construct, and for brain decoding techniques benefiting from reduced subject-specific variance in each subject's data.

Empathy-related abnormalities among women with premenstrual dysphoric disorder: clinical and functional magnetic resonance imaging study.

BACKGROUND: Empathy refers to the cognitive and emotional reactions of an individual to the experiences of another. Women with premenstrual dysphoric disorder (PMDD) report severe social difficulties during the luteal phase of their menstrual cycle. AIMS: This clinical and functional magnetic resonance imaging study aimed to explore affective and cognitive empathy in women with PMDD, during the highly symptomatic luteal phase. METHOD: Overall, 32 women with PMDD and 20 healthy controls participated in the study. The neuroimaging data were collected using a highly empathy-engaging movie. First, we characterised the synchrony of neural responses within PMDD and healthy groups, using the inter-individual correlation approach. Next, using network cohesion analysis, we compared connectivity within and between brain networks associated with affective and cognitive empathy between groups, and assessed the association of these network patterns with empathic measures. RESULTS: A consistent, although complex, picture of empathy abnormalities was found. Patients with PMDD showed decreased neural synchrony in parietal and frontal key nodes of cognitive empathy processing (theory-of-mind network), but higher neural synchrony in the anterior insula and anterior cingulate cortex, a part of the salience network, implicated in affective empathy. Positive correlations between cognitive perspective-taking scores and neural synchrony were found within the theory-of-mind network. Interestingly, during highly emotional moments, the PMDD group showed increased functional connectivity within this network. CONCLUSIONS: Similar to major depression, individuals with PMDD show enhanced affective empathy and reduced cognitive empathy. These findings echo clinical observations reported when women with PMDD have a dysregulated emotional response to negative stimuli.

Graph Analysis of the Visual Cortical Network during Naturalistic Movie Viewing Reveals Increased Integration and Decreased Segregation Following Mild TBI.

Traditional neuroimaging methods have identified alterations in brain activity patterns following mild traumatic brain injury (mTBI), particularly during rest, complex tasks, and normal vision. However, studies using graph theory to examine brain network changes in mTBI have produced varied results, influenced by the specific networks and task demands analyzed. In our study, we employed functional MRI to observe 17 mTBI patients and 54 healthy individuals as they viewed a simple, non-narrative underwater film, simulating everyday visual tasks. This approach revealed significant mTBI-related changes in network connectivity, efficiency, and organization. Specifically, the mTBI group exhibited higher overall connectivity and local network specialization, suggesting enhanced information integration without overwhelming the brain's processing capabilities. Conversely, these patients showed reduced network segregation, indicating a less compartmentalized brain function compared to healthy controls. These patterns were consistent across various visual cortex subnetworks, except in primary visual areas. Our findings highlight the potential of using naturalistic stimuli in graph-based neuroimaging to understand brain network alterations in mTBI and possibly other conditions affecting brain integration.

Default and control network connectivity dynamics track the stream of affect at multiple timescales.

In everyday life, the stream of affect results from the interaction between past experiences, expectations and the unfolding of events. How the brain represents the relationship between time and affect has been hardly explored, as it requires modeling the complexity of everyday life in the laboratory setting. Movies condense into hours a multitude of emotional responses, synchronized across subjects and characterized by temporal dynamics alike real-world experiences. Here, we use time-varying intersubject brain synchronization and real-time behavioral reports to test whether connectivity dynamics track changes in affect during movie watching. The results show that polarity and intensity of experiences relate to the connectivity of the default mode and control networks and converge in the right temporoparietal cortex. We validate these results in two experiments including four independent samples, two movies and alternative analysis workflows. Finally, we reveal chronotopic connectivity maps within the temporoparietal and prefrontal cortex, where adjacent areas preferentially encode affect at specific timescales.

The Processing of Semantic Complexity and Cospeech Gestures in Schizophrenia: A Naturalistic, Multimodal fMRI Study.

Schizophrenia is marked by aberrant processing of complex speech and gesture, which may contribute functionally to its impaired social communication. To date, extant neuroscientific studies of schizophrenia have largely investigated dysfunctional speech and gesture in isolation, and no prior research has examined how the two communicative channels may interact in more natural contexts. Here, we tested if patients with schizophrenia show aberrant neural processing of semantically complex story segments, and if speech-associated gestures (co-speech gestures) might modulate this effect. In a functional MRI study, we presented to 34 participants (16 patients and 18 matched-controls) an ecologically-valid retelling of a continuous story, performed via speech and spontaneous gestures. We split the entire story into ten-word segments, and measured the semantic complexity for each segment with idea density, a linguistic measure that is commonly used clinically to evaluate aberrant language dysfunction at the semantic level. Per segment, the presence of numbers of gestures varied (n = 0, 1, +2). Our results suggest that, in comparison to controls, patients showed reduced activation for more complex segments in the bilateral middle frontal and inferior parietal regions. Importantly, this neural aberrance was normalized in segments presented with gestures. Thus, for the first time with a naturalistic multimodal stimulation paradigm, we show that gestures reduced group differences when processing a natural story, probably by facilitating the processing of semantically complex segments of the story in schizophrenia.

Retrospective behavioral sampling (RBS): A method to effectively track the cognitive fluctuations driven by naturalistic stimulation.

Everyday experiences are dynamic, driving fluctuations across simultaneous cognitive processes. A key challenge in the study of naturalistic cognition is to disentangle the complexity of these dynamic processes, without altering the natural experience itself. Retrospective behavioral sampling (RBS) is a novel approach to model the cognitive fluctuations corresponding to the time-course of naturalistic stimulation, across a variety of cognitive dimensions. We tested the effectiveness and reliability of RBS in a web-based experiment, in which 53 participants viewed short movies and listened to a story, followed by retrospective reporting. Participants recalled their experience of 55 discrete events from the stimuli, rating their quality of memory, magnitude of surprise, intensity of negative and positive emotions, perceived importance, reflectivity state, and mental time travel. In addition, a subset of the original cohort re-rated their memory of events in a follow-up questionnaire. Results show highly replicable fluctuation patterns across distinct cognitive dimensions, thereby revealing a stimulus-driven experience that is substantially shared among individuals. Remarkably, memory ratings more than a week after stimulation resulted in an almost identical time-course of memorability as measured immediately following stimulation. In addition, idiosyncratic response patterns were preserved across different stimuli, indicating that RBS characterizes individual differences that are stimulus invariant. The current findings highlight the potential of RBS as a powerful tool for measuring dynamic processes of naturalistic cognition. We discuss the promising approach of matching RBS fluctuations with dynamic processes measured via other testing modalities, such as neuroimaging, to study the neural manifestations of naturalistic cognitive processing.

Speed Estimation for Visual Tracking Emerges Dynamically from Nonlinear Frequency Interactions.

Sensing the movement of fast objects within our visual environments is essential for controlling actions. It requires online estimation of motion direction and speed. We probed human speed representation using ocular tracking of stimuli of different statistics. First, we compared ocular responses to single drifting gratings (DGs) with a given set of spatiotemporal frequencies to broadband motion clouds (MCs) of matched mean frequencies. Motion energy distributions of gratings and clouds are point-like, and ellipses oriented along the constant speed axis, respectively. Sampling frequency space, MCs elicited stronger, less variable, and speed-tuned responses. DGs yielded weaker and more frequency-tuned responses. Second, we measured responses to patterns made of two or three components covering a range of orientations within Fourier space. Early tracking initiation of the patterns was best predicted by a linear combination of components before nonlinear interactions emerged to shape later dynamics. Inputs are supralinearly integrated along an iso-velocity line and sublinearly integrated away from it. A dynamical probabilistic model characterizes these interactions as an excitatory pooling along the iso-velocity line and inhibition along the orthogonal "scale" axis. Such crossed patterns of interaction would appropriately integrate or segment moving objects. This study supports the novel idea that speed estimation is better framed as a dynamic channel interaction organized along speed and scale axes.

Intersubject Spatial Pattern Correlations During Movie Viewing Are Stimulus-Driven and Nonuniform Across the Cortex.

A fundamental step to predicting brain activity in healthy and diseased populations is characterizing the common spatio-temporal response to a shared experience. Multivoxel pattern analysis allows us to investigate information encoding through these patterns; however, we have yet to explore local, stimulus-driven, patterns of cortical activity during naturalistic stimulation. We sought to examine these patterns with minimum interpolation-excluding functional alignment-to characterize the most basic degree of shared response between subjects. We used an unbiased analytic approach, combined with rich, naturalistic, and nonsemantic stimulation to estimate shared spatial patterns in functional magnetic resonance imaging responses across a large group. We found that meso-scale spatial patterns were shared nonuniformly across the visual cortex and represent information distinct from the shared temporal response. Shared spatial patterns were stimulus-driven, modulated by pattern size, and more sensitive to the contrast of 3D versus 2D stimulus differences than the temporal signals. Although the grand functional structure of the brain is understood to be common, these results suggest that even at a meso-scale, we share common spatial structures with anatomical alignment alone. The strength of this similarity varies across the cortex, suggesting some spatial structures are innately organized, whereas others are shaped by factors such as learning and plasticity.

Do Patients Thought to Lack Consciousness Retain the Capacity for Internal as Well as External Awareness?

It is well established that some patients, who are deemed to have disorders of consciousness, remain entirely behaviorally non-responsive and are diagnosed as being in a vegetative state, yet can nevertheless demonstrate covert awareness of their external environment by modulating their brain activity, a phenomenon known as cognitive-motor dissociation. However, the extent to which these patients retain internal awareness remains unknown. To investigate the potential for internal and external awareness in patients with chronic disorders of consciousness (DoC), we asked whether the pattern of juxtaposition between the functional time-courses of the default mode (DMN) and fronto-parietal networks, shown in healthy individuals to mediate the naturally occurring dominance switching between internal and external aspects of consciousness, was present in these patients. We used a highly engaging movie by Alfred Hitchcock to drive the recruitment of the fronto-parietal networks, including the dorsal attention (DAN) and executive control (ECN) networks, and their maximal juxtaposition to the DMN in response to the complex stimulus, relative to rest and a scrambled, meaningless movie baseline condition. We tested a control group of healthy participants (N = 13/12) and two groups of patients with disorders of consciousness, one comprised of patients who demonstrated independent, neuroimaging-based evidence of covert external awareness (N = 8), and the other of those who did not (N = 8). Similarly to the healthy controls, only the group of patients with overt and, critically, covert external awareness showed significantly heightened differentiation between the DMN and the DAN in response to movie viewing relative to their resting state time-courses, which was driven by the movie's narrative. This result suggested the presence of functional integrity in the DMN and fronto-parietal networks and their relationship to one another in patients with covert external awareness. Similar to the effect in healthy controls, these networks became more strongly juxtaposed to one another in response to movie viewing relative to the baseline conditions, suggesting the potential for internal and external awareness during complex stimulus processing. Furthermore, our results suggest that naturalistic paradigms can dissociate between groups of DoC patients with and without covert awareness based on the functional integrity of brain networks.

A communication hub for a decentralized collaboration on studying real-life cognition.

Studying the brain's behavior in situations of real-life complexity is crucial for an understanding of brain function as a whole. However, methodological difficulties and a general lack of public resources are hindering scientific progress in this domain. This channel will serve as a communication hub to collect relevant resources and curate knowledge about working paradigms, available resources, and analysis techniques.