Advancing Naturalistic Affective Science with Deep Learning.

People express their own emotions and perceive others' emotions via a variety of channels, including facial movements, body gestures, vocal prosody, and language. Studying these channels of affective behavior offers insight into both the experience and perception of emotion. Prior research has predominantly focused on studying individual channels of affective behavior in isolation using tightly controlled, non-naturalistic experiments. This approach limits our understanding of emotion in more naturalistic contexts where different channels of information tend to interact. Traditional methods struggle to address this limitation: manually annotating behavior is time-consuming, making it infeasible to do at large scale; manually selecting and manipulating stimuli based on hypotheses may neglect unanticipated features, potentially generating biased conclusions; and common linear modeling approaches cannot fully capture the complex, nonlinear, and interactive nature of real-life affective processes. In this methodology review, we describe how deep learning can be applied to address these challenges to advance a more naturalistic affective science. First, we describe current practices in affective research and explain why existing methods face challenges in revealing a more naturalistic understanding of emotion. Second, we introduce deep learning approaches and explain how they can be applied to tackle three main challenges: quantifying naturalistic behaviors, selecting and manipulating naturalistic stimuli, and modeling naturalistic affective processes. Finally, we describe the limitations of these deep learning methods, and how these limitations might be avoided or mitigated. By detailing the promise and the peril of deep learning, this review aims to pave the way for a more naturalistic affective science.

Tensorial independent component analysis reveals social and reward networks associated with major depressive disorder.

Major depressive disorder (MDD) has been associated with changes in functional brain connectivity. Yet, typical analyses of functional connectivity, such as spatial independent components analysis (ICA) for resting-state data, often ignore sources of between-subject variability, which may be crucial for identifying functional connectivity patterns associated with MDD. Typically, methods like spatial ICA will identify a single component to represent a network like the default mode network (DMN), even if groups within the data show differential DMN coactivation. To address this gap, this project applies a tensorial extension of ICA (tensorial ICA)-which explicitly incorporates between-subject variability-to identify functionally connected networks using functional MRI data from the Human Connectome Project (HCP). Data from the HCP included individuals with a diagnosis of MDD, a family history of MDD, and healthy controls performing a gambling and social cognition task. Based on evidence associating MDD with blunted neural activation to rewards and social stimuli, we predicted that tensorial ICA would identify networks associated with reduced spatiotemporal coherence and blunted social and reward-based network activity in MDD. Across both tasks, tensorial ICA identified three networks showing decreased coherence in MDD. All three networks included ventromedial prefrontal cortex, striatum, and cerebellum and showed different activation across the conditions of their respective tasks. However, MDD was only associated with differences in task-based activation in one network from the social task. Additionally, these results suggest that tensorial ICA could be a valuable tool for understanding clinical differences in relation to network activation and connectivity.

Age-related differences in ventral striatal and default mode network function during reciprocated trust.

Social relationships change across the lifespan as social networks narrow and motivational priorities shift to the present. Interestingly, aging is also associated with changes in executive function, including decision-making abilities, but it remains unclear how age-related changes in both domains interact to impact financial decisions involving other people. To study this problem, we recruited 50 human participants (Nyounger = 26, ages 18-34; Nolder = 24, ages 63-80) to play an economic trust game as the investor with three partners (friend, stranger, and computer) who played the role of investee. Investors underwent functional magnetic resonance imaging (fMRI) during the trust game while investees were seated outside of the scanner. Building on our previous work with younger adults showing both enhanced striatal responses and altered default-mode network (DMN) connectivity as a function of social closeness during reciprocated trust, we predicted that these relations would exhibit age-related differences. We found that striatal responses to reciprocated trust from friends relative to strangers and computers were blunted in older adults relative to younger adults, thus supporting our primary pre-registered hypothesis regarding social closeness. We also found that older adults exhibited enhanced DMN connectivity with the temporoparietal junction (TPJ) during reciprocated trust from friends compared to computers while younger adults exhibited the opposite pattern. Taken together, these results advance our understanding of age-related differences in sensitivity to social closeness in the context of trusting others.

Developmental Differences in Affective Representation Between Prefrontal and Subcortical Structures.

Developmental studies have identified differences in prefrontal and subcortical affective structures between children and adults, which correspond with observed cognitive and behavioral maturations from relatively simplistic emotional experiences and expressions to more nuanced, complex ones. However, developmental changes in the neural representation of emotions have not yet been well explored. It stands to reason that adults and children may demonstrate observable differences in the representation of affect within key neurological structures implicated in affective cognition. Forty-five participants (25 children; 20 adults) passively viewed positive, negative, and neutral clips from popular films while undergoing functional magnetic resonance imaging (fMRI). Using representational similarity analysis (RSA) to measure variability in neural pattern similarity, we found developmental differences between children and adults in the amygdala, nucleus accumbens (NAcc), and ventromedial prefrontal cortex (vmPFC), such that children generated less pattern similarity within subcortical structures relative to the vmPFC; a phenomenon not replicated among their older counterparts. Furthermore, children generated valence-specific differences in representational patterns across regions; these valence-specific patterns were not found in adults. These results may suggest that affective representations grow increasingly dissimilar over development as individuals mature from visceral affective responses to more evaluative analyses.

Family history of depression is associated with alterations in task-dependent connectivity between the cerebellum and ventromedial prefrontal cortex.

BACKGROUND: A family history of major depressive disorder (MDD) increases the likelihood of a future depressive episode, which itself poses a significant risk for disruptions in reward processing and social cognition. However, it is unclear whether a family history of MDD is associated with alterations in the neural circuitry underlying reward processing and social cognition. METHODS: We subdivided 279 participants from the Human Connectome Project into three groups: 71 with a lifetime history of MDD, 103 with a family history (FH) of MDD, and 105 healthy controls (HCs). We then evaluated task-based functional magnetic resonance imaging data on a social cognition and a reward processing task and found a region of the ventromedial prefrontal cortex (vmPFC) that responded to both tasks, independent of the group. To investigate whether the vmPFC shows alterations in functional connectivity between groups, we conducted psychophysiological interaction analyses using the vmPFC as a seed region. RESULTS: We found that FH (relative to HC) was associated with increased sadness scores, and MDD (relative to both FH and HC) was associated with increased sadness and MDD symptoms. Additionally, the FH group had increased vmPFC functional connectivity within the nucleus accumbens, left dorsolateral PFC, and subregions of the cerebellum relative to HC during the social cognition task. CONCLUSIONS: These findings suggest that aberrant neural mechanisms among those with a familial risk of MDD may underlie vulnerability to altered social cognition.

Functional parcellation of the default mode network: a large-scale meta-analysis.

The default mode network (DMN) consists of several regions that selectively interact to support distinct domains of cognition. Of the various sites that partake in DMN function, the posterior cingulate cortex (PCC), temporal parietal junction (TPJ), and medial prefrontal cortex (MPFC) are frequently identified as key contributors. Yet, it remains unclear whether these subcomponents of the DMN make unique contributions to specific cognitive processes and health conditions. To address this issue, we applied a meta-analytic parcellation approach used in prior work. This approach used the Neurosynth database and classification methods to quantify the association between PCC, TPJ, and MPFC activation and specific topics related to cognition and health (e.g., decision making and smoking). Our analyses replicated prior observations that the PCC, TPJ, and MPFC collectively support multiple cognitive functions such as decision making, memory, and awareness. To gain insight into the functional organization of each region, we parceled each region based on its coactivation pattern with the rest of the brain. This analysis indicated that each region could be further subdivided into functionally distinct subcomponents. Taken together, we further delineate DMN function by demonstrating the relative strengths of association among subcomponents across a range of cognitive processes and health conditions. A continued attentiveness to the specialization within the DMN allows future work to consider the nuances in sub-regional contributions necessary for healthy cognition, as well as create the potential for more targeted treatment protocols in various health conditions.