Investigating infant knowledge with representational similarity analysis.

Decades of research have pushed us closer to understanding what babies know. However, a powerful approach - representational similarity analysis (RSA) - is underused in developmental research. I discuss the strengths of this approach and what it can tell us about infant conceptual knowledge. As a case study, I focus on numerosity as a domain where RSA can make unique progress.

Functional networks in the infant brain during sleep and wake states.

Functional brain networks are assessed differently earlier versus later in development: infants are almost universally scanned asleep, whereas adults are typically scanned awake. Observed differences between infant and adult functional networks may thus reflect differing states of consciousness rather than or in addition to developmental changes. We explore this question by comparing functional networks in functional magnetic resonance imaging (fMRI) scans of infants during natural sleep and awake movie-watching. As a reference, we also scanned adults during awake rest and movie-watching. Whole-brain functional connectivity was more similar within the same state (sleep and movie in infants; rest and movie in adults) compared with across states. Indeed, a classifier trained on patterns of functional connectivity robustly decoded infant state and even generalized to adults; interestingly, a classifier trained on adult state did not generalize as well to infants. Moreover, overall similarity between infant and adult functional connectivity was modulated by adult state (stronger for movie than rest) but not infant state (same for sleep and movie). Nevertheless, the connections that drove this similarity, particularly in the frontoparietal control network, were modulated by infant state. In sum, infant functional connectivity differs between sleep and movie states, highlighting the value of awake fMRI for studying functional networks over development.

Neural effects of controllability as a key dimension of stress exposure.

Cross-species evidence suggests that the ability to exert control over a stressor is a key dimension of stress exposure that may sensitize frontostriatal-amygdala circuitry to promote more adaptive responses to subsequent stressors. The present study examined neural correlates of stressor controllability in young adults. Participants (N = 56; Mage = 23.74, range = 18-30 years) completed either the controllable or uncontrollable stress condition of the first of two novel stressor controllability tasks during functional magnetic resonance imaging (fMRI) acquisition. Participants in the uncontrollable stress condition were yoked to age- and sex-matched participants in the controllable stress condition. All participants were subsequently exposed to uncontrollable stress in the second task, which is the focus of fMRI analyses reported here. A whole-brain searchlight classification analysis revealed that patterns of activity in the right dorsal anterior insula (dAI) during subsequent exposure to uncontrollable stress could be used to classify participants' initial exposure to either controllable or uncontrollable stress with a peak of 73% accuracy. Previous experience of exerting control over a stressor may change the computations performed within the right dAI during subsequent stress exposure, shedding further light on the neural underpinnings of stressor controllability.

Face processing in the infant brain after pandemic lockdown.

The role of visual experience in the development of face processing has long been debated. We present a new angle on this question through a serendipitous study that cannot easily be repeated. Infants viewed short blocks of faces during fMRI in a repetition suppression task. The same identity was presented multiple times in half of the blocks (repeat condition) and different identities were presented once each in the other half (novel condition). In adults, the fusiform face area (FFA) tends to show greater neural activity for novel versus repeat blocks in such designs, suggesting that it can distinguish same versus different face identities. As part of an ongoing study, we collected data before the COVID-19 pandemic and after an initial local lockdown was lifted. The resulting sample of 12 infants (9-24 months) divided equally into pre- and post-lockdown groups with matching ages and data quantity/quality. The groups had strikingly different FFA responses: pre-lockdown infants showed repetition suppression (novel > repeat), whereas post-lockdown infants showed the opposite (repeat > novel), often referred to as repetition enhancement. These findings provide speculative evidence that altered visual experience during the lockdown, or other correlated environmental changes, may have affected face processing in the infant brain.

Neural event segmentation of continuous experience in human infants.

How infants experience the world is fundamental to understanding their cognition and development. A key principle of adult experience is that, despite receiving continuous sensory input, we perceive this input as discrete events. Here we investigate such event segmentation in infants and how it differs from adults. Research on event cognition in infants often uses simplified tasks in which (adult) experimenters help solve the segmentation problem for infants by defining event boundaries or presenting discrete actions/vignettes. This presupposes which events are experienced by infants and leaves open questions about the principles governing infant segmentation. We take a different, data-driven approach by studying infant event segmentation of continuous input. We collected whole-brain functional MRI (fMRI) data from awake infants (and adults, for comparison) watching a cartoon and used a hidden Markov model to identify event states in the brain. We quantified the existence, timescale, and organization of multiple-event representations across brain regions. The adult brain exhibited a known hierarchical gradient of event timescales, from shorter events in early visual regions to longer events in later visual and associative regions. In contrast, the infant brain represented only longer events, even in early visual regions, with no timescale hierarchy. The boundaries defining these infant events only partially overlapped with boundaries defined from adult brain activity and behavioral judgments. These findings suggest that events are organized differently in infants, with longer timescales and more stable neural patterns, even in sensory regions. This may indicate greater temporal integration and reduced temporal precision during dynamic, naturalistic perception.

Context Matters: Recovering Human Semantic Structure from Machine Learning Analysis of Large-Scale Text Corpora.

Applying machine learning algorithms to automatically infer relationships between concepts from large-scale collections of documents presents a unique opportunity to investigate at scale how human semantic knowledge is organized, how people use it to make fundamental judgments ("How similar are cats and bears?"), and how these judgments depend on the features that describe concepts (e.g., size, furriness). However, efforts to date have exhibited a substantial discrepancy between algorithm predictions and human empirical judgments. Here, we introduce a novel approach to generating embeddings for this purpose motivated by the idea that semantic context plays a critical role in human judgment. We leverage this idea by constraining the topic or domain from which documents used for generating embeddings are drawn (e.g., referring to the natural world vs. transportation apparatus). Specifically, we trained state-of-the-art machine learning algorithms using contextually-constrained text corpora (domain-specific subsets of Wikipedia articles, 50+ million words each) and showed that this procedure greatly improved predictions of empirical similarity judgments and feature ratings of contextually relevant concepts. Furthermore, we describe a novel, computationally tractable method for improving predictions of contextually-unconstrained embedding models based on dimensionality reduction of their internal representation to a small number of contextually relevant semantic features. By improving the correspondence between predictions derived automatically by machine learning methods using vast amounts of data and more limited, but direct empirical measurements of human judgments, our approach may help leverage the availability of online corpora to better understand the structure of human semantic representations and how people make judgments based on those.

Retinotopic organization of visual cortex in human infants.

Vision develops rapidly during infancy, yet how visual cortex is organized during this period is unclear. In particular, it is unknown whether functional maps that organize the mature adult visual cortex are present in the infant striate and extrastriate cortex. Here, we test the functional maturity of infant visual cortex by performing retinotopic mapping with functional magnetic resonance imaging (fMRI). Infants aged 5-23 months had retinotopic maps, with alternating preferences for vertical and horizontal meridians indicating the boundaries of visual areas V1 to V4 and an orthogonal gradient of preferences from high to low spatial frequencies. The presence of multiple visual maps throughout visual cortex in infants indicates a greater maturity of extrastriate cortex than previously appreciated. The areas showed subtle age-related fine-tuning, suggesting that early maturation undergoes continued refinement. This early maturation of area boundaries and tuning may scaffold subsequent developmental changes.

Evidence of hippocampal learning in human infants.

The hippocampus is essential for human memory.1 The protracted maturation of memory capacities from infancy through early childhood2-4 is thus often attributed to hippocampal immaturity.5-7 The hippocampus of human infants has been characterized in terms of anatomy,8,9 but its function has never been tested directly because of technical challenges.10,11 Here, we use recently developed methods for task-based fMRI in awake human infants12 to test the hypothesis that the infant hippocampus supports statistical learning.13-15 Hippocampal activity increased with exposure to visual sequences of objects when the temporal order contained regularities to be learned, compared to when the order was random. Despite the hippocampus doubling in anatomical volume across infancy, learning-related functional activity bore no relationship to age. This suggests that the hippocampus is recruited for statistical learning at the youngest ages in our sample, around 3 months. Within the hippocampus, statistical learning was clearer in anterior than posterior divisions. This is consistent with the theory that statistical learning occurs in the monosynaptic pathway,16 which is more strongly represented in the anterior hippocampus.17,18 The monosynaptic pathway develops earlier than the trisynaptic pathway, which is linked to episodic memory,19,20 raising the possibility that the infant hippocampus participates in statistical learning before it forms durable memories. Beyond the hippocampus, the medial prefrontal cortex showed statistical learning, consistent with its role in adult memory integration21 and generalization.22 These results suggest that the hippocampus supports the vital ability of infants to extract the structure of their environment through experience.

Attention recruits frontal cortex in human infants.

Young infants learn about the world by overtly shifting their attention to perceptually salient events. In adults, attention recruits several brain regions spanning the frontal and parietal lobes. However, it is unclear whether these regions are sufficiently mature in infancy to support attention and, more generally, how infant attention is supported by the brain. We used event-related functional magnetic resonance imaging (fMRI) in 24 sessions from 20 awake behaving infants 3 mo to 12 mo old while they performed a child-friendly attentional cuing task. A target was presented to either the left or right of the infant's fixation, and offline gaze coding was used to measure the latency with which they saccaded to the target. To manipulate attention, a brief cue was presented before the target in three conditions: on the same side as the upcoming target (valid), on the other side (invalid), or on both sides (neutral). All infants were faster to look at the target on valid versus invalid trials, with valid faster than neutral and invalid slower than neutral, indicating that the cues effectively captured attention. We then compared the fMRI activity evoked by these trial types. Regions of adult attention networks activated more strongly for invalid than valid trials, particularly frontal regions. Neither behavioral nor neural effects varied by infant age within the first year, suggesting that these regions may function early in development to support the orienting of attention. Together, this furthers our mechanistic understanding of how the infant brain controls the allocation of attention.

BrainIAK: The Brain Imaging Analysis Kit.

Functional magnetic resonance imaging (fMRI) offers a rich source of data for studying the neural basis of cognition. Here, we describe the Brain Imaging Analysis Kit (BrainIAK), an open-source, free Python package that provides computationally optimized solutions to key problems in advanced fMRI analysis. A variety of techniques are presently included in BrainIAK: intersubject correlation (ISC) and intersubject functional connectivity (ISFC), functional alignment via the shared response model (SRM), full correlation matrix analysis (FCMA), a Bayesian version of representational similarity analysis (BRSA), event segmentation using hidden Markov models, topographic factor analysis (TFA), inverted encoding models (IEMs), an fMRI data simulator that uses noise characteristics from real data (fmrisim), and some emerging methods. These techniques have been optimized to leverage the efficiencies of high-performance compute (HPC) clusters, and the same code can be se amlessly transferred from a laptop to a cluster. For each of the aforementioned techniques, we describe the data analysis problem that the technique is meant to solve and how it solves that problem; we also include an example Jupyter notebook for each technique and an annotated bibliography of papers that have used and/or described that technique. In addition to the sections describing various analysis techniques in BrainIAK, we have included sections describing the future applications of BrainIAK to real-time fMRI, tutorials that we have developed and shared online to facilitate learning the techniques in BrainIAK, computational innovations in BrainIAK, and how to contribute to BrainIAK. We hope that this manuscript helps readers to understand how BrainIAK might be useful in their research.

Emergence and organization of adult brain function throughout child development.

Adult cognitive neuroscience has guided the study of human brain development by identifying regions associated with cognitive functions at maturity. The activity, connectivity, and structure of a region can be compared across ages to characterize the developmental trajectory of the corresponding function. However, developmental differences may reflect both the maturation of the function and also its organization across the brain. That is, a function may be present in children but supported by different brain regions, leading its maturity to be underestimated. Here we test the presence, maturity, and localization of adult functions in children using shared response modeling, a machine learning approach for functional alignment. After learning a lower-dimensional feature space from fMRI activity as adults watched a movie, we translated these shared features into the anatomical brain space of children 3-12 years old. To evaluate functional maturity, we correlated this reconstructed activity with children's actual fMRI activity as they watched the same movie. We found reliable correlations throughout cortex, even in the youngest children. The strength of the correlation in the precuneus, inferior frontal gyrus, and lateral occipital cortex predicted chronological age. These age-related changes were driven by three types of developmental trajectories: emergence from absence to presence, consistency in anatomical expression, and reorganization from one anatomical region to another. We also found evidence that the processing of pain-related events in the movie underwent reorganization across childhood. This data-driven, naturalistic approach provides a new perspective on the development of functional neuroanatomy throughout childhood.

Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain.

The extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model, as well as object representations, are more widely distributed across the brain than previously acknowledged and that functional searchlight can improve model-based similarity and decoding accuracy. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.

Facilitating open-science with realistic fMRI simulation: validation and application.

With advances in methods for collecting and analyzing fMRI data, there is a concurrent need to understand how to reliably evaluate and optimally use these methods. Simulations of fMRI data can aid in both the evaluation of complex designs and the analysis of data. We present fmrisim, a new Python package for standardized, realistic simulation of fMRI data. This package is part of BrainIAK: a recently released open-source Python toolbox for advanced neuroimaging analyses. We describe how to use fmrisim to extract noise properties from real fMRI data and then create a synthetic dataset with matched noise properties and a user-specified signal. We validate the noise generated by fmrisim to show that it can approximate the noise properties of real data. We further show how fmrisim can help researchers find the optimal design in terms of power. The fmrisim package holds promise for improving the design of fMRI experiments, which may facilitate both the pre-registration of such experiments as well as the analysis of fMRI data.

BrainIAK tutorials: User-friendly learning materials for advanced fMRI analysis.

Advanced brain imaging analysis methods, including multivariate pattern analysis (MVPA), functional connectivity, and functional alignment, have become powerful tools in cognitive neuroscience over the past decade. These tools are implemented in custom code and separate packages, often requiring different software and language proficiencies. Although usable by expert researchers, novice users face a steep learning curve. These difficulties stem from the use of new programming languages (e.g., Python), learning how to apply machine-learning methods to high-dimensional fMRI data, and minimal documentation and training materials. Furthermore, most standard fMRI analysis packages (e.g., AFNI, FSL, SPM) focus on preprocessing and univariate analyses, leaving a gap in how to integrate with advanced tools. To address these needs, we developed BrainIAK (brainiak.org), an open-source Python software package that seamlessly integrates several cutting-edge, computationally efficient techniques with other Python packages (e.g., Nilearn, Scikit-learn) for file handling, visualization, and machine learning. To disseminate these powerful tools, we developed user-friendly tutorials (in Jupyter format; https://brainiak.org/tutorials/) for learning BrainIAK and advanced fMRI analysis in Python more generally. These materials cover techniques including: MVPA (pattern classification and representational similarity analysis); parallelized searchlight analysis; background connectivity; full correlation matrix analysis; inter-subject correlation; inter-subject functional connectivity; shared response modeling; event segmentation using hidden Markov models; and real-time fMRI. For long-running jobs or large memory needs we provide detailed guidance on high-performance computing clusters. These notebooks were successfully tested at multiple sites, including as problem sets for courses at Yale and Princeton universities and at various workshops and hackathons. These materials are freely shared, with the hope that they become part of a pool of open-source software and educational materials for large-scale, reproducible fMRI analysis and accelerated discovery.

Feasibility of topological data analysis for event-related fMRI.

Recent fMRI research shows that perceptual and cognitive representations are instantiated in high-dimensional multivoxel patterns in the brain. However, the methods for detecting these representations are limited. Topological data analysis (TDA) is a new approach, based on the mathematical field of topology, that can detect unique types of geometric features in patterns of data. Several recent studies have successfully applied TDA to study various forms of neural data; however, to our knowledge, TDA has not been successfully applied to data from event-related fMRI designs. Event-related fMRI is very common but limited in terms of the number of events that can be run within a practical time frame and the effect size that can be expected. Here, we investigate whether persistent homology-a popular TDA tool that identifies topological features in data and quantifies their robustness-can identify known signals given these constraints. We use fmrisim, a Python-based simulator of realistic fMRI data, to assess the plausibility of recovering a simple topological representation under a variety of conditions. Our results suggest that persistent homology can be used under certain circumstances to recover topological structure embedded in realistic fMRI data simulations.

Complexity can facilitate visual and auditory perception.

Visual and auditory inputs vary in complexity. For example, driving in a city versus the country or listening to the radio versus not are experiences that differ in complexity. How does such complexity impact perception? One possibility is that complex stimuli demand resources that exceed attentional or working memory capacities, reducing sensitivity to perceptual changes. Alternatively, complexity may allow for richer and more distinctive representations, increasing such sensitivity. We performed five experiments to test the nature of the relationship between complexity and perceptual sensitivity during movie clip viewing. Experiment 1 revealed higher sensitivity to global changes in audio or video streams for clips with greater complexity, defined both subjectively (judgments by independent coders) and objectively (information-theoretic redundancy). Experiment 2 replicated this finding but found no evidence that it resulted from complexity drawing attention. Experiment 3 provided a boundary condition by showing that change detection was unaffected by complexity when the changes were superimposed on, rather than dispersed throughout, the clips. Experiment 4 suggested that the effect of complexity, at least when defined objectively, was present without the working memory demands of the preceding experiments. Experiment 5 suggested that complexity led to richer representations of the clips, as reflected in enhanced long-term memory. Collectively, these findings show that, despite increasing informational load, complexity can serve to ground and facilitate perceptual sensitivity. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Infant fMRI: A Model System for Cognitive Neuroscience.

Our understanding of the typical human brain has benefitted greatly from studying different kinds of brains and their associated behavioral repertoires, including animal models and neuropsychological patients. This same comparative perspective can be applied to early development - the environment, behavior, and brains of infants provide a model system for understanding how the mature brain works. This approach requires noninvasive methods for measuring brain function in awake, behaving infants. fMRI is becoming increasingly viable for this purpose, with the unique ability to precisely measure the entire brain, including both cortical and subcortical structures. Here we discuss potential lessons from infant fMRI for several domains of adult cognition and consider the challenges of conducting such research and how they might be mitigated.