Data-driven analysis of facial thermal responses and multimodal physiological consistency among subjects.

Facial infra-red imaging (IRI) is a contact-free technique complimenting the traditional psychophysiological measures to characterize physiological profile. However, its full potential in affective research is arguably unmet due to the analytical challenges it poses. Here we acquired facial IRI data, facial expressions and traditional physiological recordings (heart rate and skin conductance) from healthy human subjects whilst they viewed a 20-min-long unedited emotional movie. We present a novel application of motion correction and the results of spatial independent component analysis of the thermal data. Three distinct spatial components are recovered associated with the nose, the cheeks and respiration. We first benchmark this methodology against a traditional nose-tip region-of-interest based technique showing an expected similarity of signals extracted by these methods. We then show significant correlation of all the physiological responses across subjects, including the thermal signals, suggesting common dynamic shifts in emotional state induced by the movie. In sum, this study introduces an innovative approach to analyse facial IRI data and highlights the potential of thermal imaging to robustly capture emotion-related changes induced by ecological stimuli.

Detecting changes in facial temperature induced by a sudden auditory stimulus based on deep learning-assisted face tracking.

Thermal Imaging (Infrared-Imaging-IRI) is a promising new technique for psychophysiological research and application. Unlike traditional physiological measures (like skin conductance and heart rate), it is uniquely contact-free, substantially enhancing its ecological validity. Investigating facial regions and subsequent reliable signal extraction from IRI data is challenging due to head motion artefacts. Exploiting its potential thus depends on advances in analytical methods. Here, we developed a novel semi-automated thermal signal extraction method employing deep learning algorithms for facial landmark identification. We applied this method to physiological responses elicited by a sudden auditory stimulus, to determine if facial temperature changes induced by a stimulus of a loud sound can be detected. We compared thermal responses with psycho-physiological sensor-based tools of galvanic skin response (GSR) and electrocardiography (ECG). We found that the temperatures of selected facial regions, particularly the nose tip, significantly decreased after the auditory stimulus. Additionally, this response was quite rapid at around 4-5 seconds, starting less than 2 seconds following the GSR changes. These results demonstrate that our methodology offers a sensitive and robust tool to capture facial physiological changes with minimal manual intervention and manual pre-processing of signals. Newer methodological developments for reliable temperature extraction promise to boost IRI use as an ecologically-valid technique in social and affective neuroscience.

Distinct Cerebellar Contributions to Cognitive-Perceptual Dynamics During Natural Viewing.

The crucial role of the cerebellum in motor learning and coordination is very well known. Considerable interest has recently shifted toward its contribution to nonmotor tasks, such as working memory, emotion, and language. However, the cognitive role and functional subdivisions of the cerebellum, particularly in dynamic, ecologically realistic contexts, are not yet established. By analyzing functional neuroimaging data acquired while participants viewed a short dramatic movie, we found that posterior and inferior cerebellar regions are reliably engaged in dynamic perceptual and affective processes with no explicit motor component. These cerebellar regions show significant relevance to visual salience and unexpected turning points of the movie. Our results demonstrate that distinct functional subdivisions of the cerebellum are robustly engaged in real-life cognitive processes, playing specific roles through a dynamic interaction with higher order regions in the cerebral cortex.

Inter-subject Functional Correlation Reveal a Hierarchical Organization of Extrinsic and Intrinsic Systems in the Brain.

The brain is constantly monitoring and integrating both cues from the external world and signals generated intrinsically. These extrinsically and intrinsically-driven neural processes are thought to engage anatomically distinct regions, which are thought to constitute the extrinsic and intrinsic systems of the brain. While the specialization of extrinsic and intrinsic system is evident in primary and secondary sensory cortices, a systematic mapping of the whole brain remains elusive. Here, we characterized the extrinsic and intrinsic functional activities in the brain during naturalistic movie-viewing. Using a novel inter-subject functional correlation (ISFC) analysis, we found that the strength of ISFC shifts along the hierarchical organization of the brain. Primary sensory cortices appear to have strong inter-subject functional correlation, consistent with their role in processing exogenous information, while heteromodal regions that attend to endogenous processes have low inter-subject functional correlation. Those brain systems with higher intrinsic tendency show greater inter-individual variability, likely reflecting the aspects of brain connectivity architecture unique to individuals. Our study presents a novel framework for dissecting extrinsically- and intrinsically-driven processes, as well as examining individual differences in brain function during naturalistic stimulation.

Test-retest reliability of functional connectivity networks during naturalistic fMRI paradigms.

Functional connectivity analysis has become a powerful tool for probing the human brain function and its breakdown in neuropsychiatry disorders. So far, most studies adopted resting-state paradigm to examine functional connectivity networks in the brain, thanks to its low demand and high tolerance that are essential for clinical studies. However, the test-retest reliability of resting-state connectivity measures is moderate, potentially due to its low behavioral constraint. On the other hand, naturalistic neuroimaging paradigms, an emerging approach for cognitive neuroscience with high ecological validity, could potentially improve the reliability of functional connectivity measures. To test this hypothesis, we characterized the test-retest reliability of functional connectivity measures during a natural viewing condition, and benchmarked it against resting-state connectivity measures acquired within the same functional magnetic resonance imaging (fMRI) session. We found that the reliability of connectivity and graph theoretical measures of brain networks is significantly improved during natural viewing conditions over resting-state conditions, with an average increase of almost 50% across various connectivity measures. Not only sensory networks for audio-visual processing become more reliable, higher order brain networks, such as default mode and attention networks, but also appear to show higher reliability during natural viewing. Our results support the use of natural viewing paradigms in estimating functional connectivity of brain networks, and have important implications for clinical application of fMRI. Hum Brain Mapp 38:2226-2241, 2017. © 2017 Wiley Periodicals, Inc.

The integration of the internal and external milieu in the insula during dynamic emotional experiences.

Whilst external events trigger emotional responses, interoception (the perception of internal physiological states) is fundamental to core emotional experience. By combining high resolution functional neuroimaging with concurrent physiological recordings, we investigated the neural mechanisms of interoceptive integration during free listening to an emotionally salient audio film. We found that cardiac activity, a key interoceptive signal, was robustly synchronised across participants and centrally represented in the posterior insula. Effective connectivity analysis revealed that the anterior insula, specifically tuned to the emotionally salient moments of the audio stream, serves as an integration hub of interoceptive processing: interoceptive states represented in the posterior insula are integrated with exteroceptive representations by the anterior insula to highlight these emotionally salient moments. Our study for the first time demonstrates the insular hierarchy for interoceptive processing during natural emotional experience. These findings provide an ecologically-valid framework for elucidating the neural underpinnings of emotional deficits in neuropsychiatric disorders.