A shared model-based linguistic space for transmitting our thoughts from brain to brain in natural conversations.

Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker's brain before word articulation and rapidly re-emerges in the listener's brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.

A shared linguistic space for transmitting our thoughts from brain to brain in natural conversations.

Effective communication hinges on a mutual understanding of word meaning in different contexts. The embedding space learned by large language models can serve as an explicit model of the shared, context-rich meaning space humans use to communicate their thoughts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We demonstrate that the linguistic embedding space can capture the linguistic content of word-by-word neural alignment between speaker and listener. Linguistic content emerged in the speaker's brain before word articulation, and the same linguistic content rapidly reemerged in the listener's brain after word articulation. These findings establish a computational framework to study how human brains transmit their thoughts to one another in real-world contexts.

Shared computational principles for language processing in humans and deep language models.

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models generate appropriate linguistic responses in a given context. In the current study, nine participants listened to a 30-min podcast while their brain responses were recorded using electrocorticography (ECoG). We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process the same natural narrative: (1) both are engaged in continuous next-word prediction before word onset; (2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise; (3) both rely on contextual embeddings to represent words in natural contexts. Together, our findings suggest that autoregressive DLMs provide a new and biologically feasible computational framework for studying the neural basis of language.

The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension.

The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

Neural alignment predicts learning outcomes in students taking an introduction to computer science course.

Despite major advances in measuring human brain activity during and after educational experiences, it is unclear how learners internalize new content, especially in real-life and online settings. In this work, we introduce a neural approach to predicting and assessing learning outcomes in a real-life setting. Our approach hinges on the idea that successful learning involves forming the right set of neural representations, which are captured in canonical activity patterns shared across individuals. Specifically, we hypothesized that learning is mirrored in neural alignment: the degree to which an individual learner's neural representations match those of experts, as well as those of other learners. We tested this hypothesis in a longitudinal functional MRI study that regularly scanned college students enrolled in an introduction to computer science course. We additionally scanned graduate student experts in computer science. We show that alignment among students successfully predicts overall performance in a final exam. Furthermore, within individual students, we find better learning outcomes for concepts that evoke better alignment with experts and with other students, revealing neural patterns associated with specific learned concepts in individuals.

Infant and Adult Brains Are Coupled to the Dynamics of Natural Communication.

Infancy is the foundational period for learning from adults, and the dynamics of the social environment have long been considered central to children's development. Here, we reveal a novel, naturalistic approach for studying live interactions between infants and adults. Using functional near-infrared spectroscopy (fNIRS), we simultaneously and continuously measured the brains of infants (N = 18; 9-15 months of age) and an adult while they communicated and played with each other. We found that time-locked neural coupling within dyads was significantly greater when dyad members interacted with each other than with control individuals. In addition, we characterized the dynamic relationship between neural activation and the moment-to-moment fluctuations of mutual gaze, joint attention to objects, infant emotion, and adult speech prosody. This investigation advances what is currently known about how the brains and behaviors of infants both shape and reflect those of adults during real-life communication.

Temperament and peer problems from early to middle childhood: Gene-environment correlations with negative emotionality and sociability.

Based in a transactional framework in which children's own characteristics and the social environment influence each other to produce individual differences in social adjustment, we investigated relationships between children's peer problems and their temperamental characteristics, using a longitudinal and genetically informed study of 939 pairs of Israeli twins followed from early to middle childhood (ages 3, 5, and 6.5). Peer problems were moderately stable within children over time, such that children who appeared to have more peer problems at age 3 tended to have also more peer problems at age 6.5. Children's temperament accounted for 10%-22% of the variance in their peer problems measured at the same age and for 2%-7% of the variance longitudinally. It is important that genetic factors accounted for the association between temperament and peer problems and were in line with a gene-environment correlation process, providing support for the proposal that biologically predisposed characteristics, particularly negative emotionality and sociability, have an influence on children's early experiences of peer problems. The results highlight the need for early and continuous interventions that are specifically tailored to address the interpersonal difficulties of children with particular temperamental profiles.