Information flow across the cortical timescale hierarchy during narrative construction.

When listening to spoken narratives, we must integrate information over multiple, concurrent timescales, building up from words to sentences to paragraphs to a coherent narrative. Recent evidence suggests that the brain relies on a chain of hierarchically organized areas with increasing temporal receptive windows to process naturalistic narratives. We hypothesized that the structure of this cortical processing hierarchy should result in an observable sequence of response lags between networks comprising the hierarchy during narrative comprehension. This study uses functional MRI to estimate the response lags between functional networks during narrative comprehension. We use intersubject cross-correlation analysis to capture network connectivity driven by the shared stimulus. We found a fixed temporal sequence of response lags-on the scale of several seconds-starting in early auditory areas, followed by language areas, the attention network, and lastly the default mode network. This gradient is consistent across eight distinct stories but absent in data acquired during rest or using a scrambled story stimulus, supporting our hypothesis that narrative construction gives rise to internetwork lags. Finally, we build a simple computational model for the neural dynamics underlying the construction of nested narrative features. Our simulations illustrate how the gradual accumulation of information within the boundaries of nested linguistic events, accompanied by increased activity at each level of the processing hierarchy, can give rise to the observed lag gradient.

Relating the Past with the Present: Information Integration and Segregation during Ongoing Narrative Processing.

This study examined how the brain dynamically updates event representations by integrating new information over multiple minutes while segregating irrelevant input. A professional writer custom-designed a narrative with two independent storylines, interleaving across minute-long segments (ABAB). In the last (C) part, characters from the two storylines meet and their shared history is revealed. Part C is designed to induce the spontaneous recall of past events, upon the recurrence of narrative motifs from A/B, and to shed new light on them. Our fMRI results showed storyline-specific neural patterns, which were reinstated (i.e., became more active) during storyline transitions. This effect increased along the processing timescale hierarchy, peaking in the default mode network. Similarly, the neural reinstatement of motifs was found during Part C. Furthermore, participants showing stronger motif reinstatement performed better in integrating A/B and C events, demonstrating the role of memory reactivation in information integration over intervening irrelevant events.

Adaptation of the human visual system to the statistics of letters and line configurations.

By adulthood, literate humans have been exposed to millions of visual scenes and pages of text. Does the human visual system become attuned to the statistics of its inputs? Using functional magnetic resonance imaging, we examined whether the brain responses to line configurations are proportional to their natural-scene frequency. To further distinguish prior cortical competence from adaptation induced by learning to read, we manipulated whether the selected configurations formed letters and whether they were presented on the horizontal meridian, the familiar location where words usually appear, or on the vertical meridian. While no natural-scene frequency effect was observed, we observed letter-status and letter frequency effects on bilateral occipital activation, mainly for horizontal stimuli. The findings suggest a reorganization of the visual pathway resulting from reading acquisition under genetic and connectional constraints. Even early retinotopic areas showed a stronger response to letters than to rotated versions of the same shapes, suggesting an early visual tuning to large visual features such as letters.

How a speaker herds the audience: Multi-brain neural convergence over time during naturalistic storytelling.

Storytelling-an ancient way for humans to share individual experiences with others-has been found to induce neural synchronization among listeners. In our exploration of the dynamic fluctuations in listener-listener (LL) coupling throughout stories, we uncover a significant correlation between LL and lag-speaker-listener (lag-SL) couplings over time. Using the analogy of neural pattern (dis)similarity as distances between participants, we term this phenomenon the "herding effect": like a shepherd guiding a group of sheep, the more closely listeners follow the speaker's prior brain activity patterns (higher lag-SL similarity), the more tightly they cluster together (higher LL similarity). This herding effect is particularly pronounced in brain regions where neural synchronization among listeners tracks with behavioral ratings of narrative engagement, highlighting the mediating role of narrative content in the observed multi-brain neural coupling dynamics. By integrating LL and SL neural couplings, this study illustrates how unfolding stories shape a dynamic multi-brain functional network and how the configuration of this network may be associated with moment-by-moment efficacy of communication.

How a speaker herds the audience: multibrain neural convergence over time during naturalistic storytelling.

Storytelling-an ancient way for humans to share individual experiences with others-has been found to induce neural alignment among listeners. In exploring the dynamic fluctuations in listener-listener (LL) coupling throughout stories, we uncover a significant correlation between LL coupling and lagged speaker-listener (lag-SL) coupling over time. Using the analogy of neural pattern (dis)similarity as distances between participants, we term this phenomenon the "herding effect." Like a shepherd guiding a group of sheep, the more closely listeners mirror the speaker's preceding brain activity patterns (higher lag-SL similarity), the more tightly they cluster (higher LL similarity). This herding effect is particularly pronounced in brain regions where neural alignment among listeners tracks with moment-by-moment behavioral ratings of narrative content engagement. By integrating LL and SL neural coupling, this study reveals a dynamic, multibrain functional network between the speaker and the audience, with the unfolding narrative content playing a mediating role in network configuration.

The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension.

The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

Cortical encoding of linguistic constituent with and without morphosyntactic cues.

This study examined the brain areas involved in combining words into larger units when there are few or no morphosyntactic cues. We manipulated constituent length in word strings of the same length under two conditions: Mandarin sentence, which had sparse morphosyntactic cues, and nominal phrase that had no morphosyntactic cues [e.g., ((honey mustard) (chicken burger))]. Contrasting sentences to word lists revealed a network that largely overlapped with the one reported in languages with rich morphosyntactic cues, including left IFGorb/IFGtri and areas along left STG/STS. Both conditions showed increased activation in left IFGtri/IFGorb in functional ROIs defined based on previous study in sentence processing, while the nominal phrases additionally revealed a constituent length effect in bilateral dorsal IFGtri, left IFGoper, left pMTG/pSTG, left IPL, and several subcortical areas, which might reflect an increased reliance on semantic and pragmatic information. Moreover, in upper left IFGtri/IFGoper and left thalamus/caudate, this effect increased with the participants' tendency to combine nouns into phrases. The absence of syntactic constraints on linguistic composition might highlight individual differences in cognitive control, which helps to integrate non-syntactic information.

Differential brain mechanisms during reading human vs. machine translated fiction and news texts.

Few neuroimaigng studies on reading comprehension have been conducted under natural reading settings. In this study, we showed texts presented in a natural way during functional MRI (fMRI) measurements to reveal brain areas sensitive to reading comprehension. Specifically, this paradigm independently manipulated two holistic features of article style: text genre and translation style, a qualitative index of how typical word choices and arrangements are made in daily use of the language. Specifically, articles from The New York Times (news) and Reader's Digest (fiction) translated from English to Mandarin Chinese either by human experts or machine (Google Translate) were used to investigate the correlation of brain activity across participants during article reading. We found that bi-hemispheric visual cortex, precuneus, and occipito-parietal junction show significantly correlated hemodynamics across participants regardless of translation style and article genre. Compared to machine translation, reading human expert translation elicited more reliable fMRI signals across participants at precuneus, potentially because narrative representations and contents can be coherently presented over tens of seconds. We also found significantly stronger inter-subject correlated fMRI signals at temporal poles and fusiform gyri in fiction reading than in news reading. This may be attributed to more stable empathy processing across participants in fiction reading. The degree of stability of brain responses across subjects at extra-linguistic areas was found correlated with subjective rating on the text fluency. The functional connectivity between these areas was modulated by text genre and translation style. Taken together, our imaging results suggested stable and selective neural substrates associated with comprehending holistic features of written narratives.

The Neural Substrates Underlying the Implementation of Phonological Rule in Lexical Tone Production: An fMRI Study of the Tone 3 Sandhi Phenomenon in Mandarin Chinese.

This study examined the neural substrates underlying the implementation of phonological rule in lexical tone by the Tone 3 sandhi phenomenon in Mandarin Chinese. Tone 3 sandhi is traditionally described as the substitution of Tone 3 with Tone 2 when followed by another Tone 3 (33 →23) during speech production. Tone 3 sandhi enables the examination of tone processing in the phonological level with the least involvement of segments. Using the fMRI technique, we measured brain activations corresponding to the monosyllable and disyllable sequences of the four Chinese lexical tones, while manipulating the requirement on overt oral response. The application of Tone 3 sandhi to disyllable sequence of Tone 3 was confirmed by our behavioral results. Larger brain responses to overtly produced disyllable Tone 3 (33 > 11, 22, and 44) were found in right posterior IFG by both whole-brain and ROI analyses. We suggest that the right IFG was responsible for the processing of Tone 3 sandhi. Intense temporo-frontal interaction is needed in speech production for self-monitoring. The involvement of the right IFG in tone production might result from its interaction with the right auditory cortex, which is known to specialize in pitch. Future studies using tools with better temporal resolutions are needed to illuminate the dynamic interaction between the right inferior frontal regions and the left-lateralized language network in tone languages.