Early-emerging combinatorial thought: Human infants flexibly combine kind and quantity concepts.

Combinatorial thought, or the ability to combine a finite set of concepts into a myriad of complex ideas and knowledge structures, is the key to the productivity of the human mind and underlies communication, science, technology, and art. Despite the importance of combinatorial thought for human cognition and culture, its developmental origins remain unknown. To address this, we tested whether 12-mo-old infants (N = 60), who cannot yet speak and only understand a handful of words, can combine quantity and kind concepts activated by verbal input. We proceeded in two steps: first, we taught infants two novel labels denoting quantity (e.g., "mize" for 1 item; "padu" for 2 items, Experiment 1). Then, we assessed whether they could combine quantity and kind concepts upon hearing complex expressions comprising their labels (e.g., "padu duck", Experiments 2-3). At test, infants viewed four different sets of objects (e.g., 1 duck, 2 ducks, 1 ball, 2 balls) while being presented with the target phrase (e.g., "padu duck") naming one of them (e.g., 2 ducks). They successfully retrieved and combined on-line the labeled concepts, as evidenced by increased looking to the named sets but not to distractor sets. Our results suggest that combinatorial processes for building complex representations are available by the end of the first year of life. The infant mind seems geared to integrate concepts in novel productive ways. This ability may be a precondition for deciphering the ambient language(s) and building abstract models of experience that enable fast and flexible learning.

Curriculum learning for human compositional generalization.

Generalization (or transfer) is the ability to repurpose knowledge in novel settings. It is often asserted that generalization is an important ingredient of human intelligence, but its extent, nature, and determinants have proved controversial. Here, we examine this ability with a paradigm that formalizes the transfer learning problem as one of recomposing existing functions to solve unseen problems. We find that people can generalize compositionally in ways that are elusive for standard neural networks and that human generalization benefits from training regimes in which items are axis aligned and temporally correlated. We describe a neural network model based around a Hebbian gating process that can capture how human generalization benefits from different training curricula. We additionally find that adult humans tend to learn composable functions asynchronously, exhibiting discontinuities in learning that resemble those seen in child development.

Seeds of language-like generativity in bird call combinations.

Language is unbounded in its generativity, enabling the flexible combination of words into novel sentences. Critically, these constructions are intelligible to others due to our ability to derive a sentence's compositional meaning from the semantic relationships among its components. Some animals also concatenate meaningful calls into compositional-like combinations to communicate more complex information. However, these combinations are structurally highly stereotyped, suggesting a bounded system of holistically perceived signals that impedes the processing of novel variants. Using long-term data and playback experiments on pied babblers, we demonstrate that, despite production stereotypy, they can nevertheless process structurally modified and novel combinations of their calls, demonstrating a capacity for deriving meaning compositionally. Furthermore, differential responses to artificial combinations by fledglings suggest that this compositional sensitivity is acquired ontogenetically. Our findings demonstrate animal combinatorial systems can be flexible at the perceptual level and that such perceptual flexibility may represent a precursor of language-like generativity.

Statistical challenges in longitudinal microbiome data analysis.

The microbiome is a complex and dynamic community of microorganisms that co-exist interdependently within an ecosystem, and interact with its host or environment. Longitudinal studies can capture temporal variation within the microbiome to gain mechanistic insights into microbial systems; however, current statistical methods are limited due to the complex and inherent features of the data. We have identified three analytical objectives in longitudinal microbial studies: (1) differential abundance over time and between sample groups, demographic factors or clinical variables of interest; (2) clustering of microorganisms evolving concomitantly across time and (3) network modelling to identify temporal relationships between microorganisms. This review explores the strengths and limitations of current methods to fulfill these objectives, compares different methods in simulation and case studies for objectives (1) and (2), and highlights opportunities for further methodological developments. R tutorials are provided to reproduce the analyses conducted in this review.

Emergence of a compositional neural code for written words: Recycling of a convolutional neural network for reading.

The visual word form area (VWFA) is a region of human inferotemporal cortex that emerges at a fixed location in the occipitotemporal cortex during reading acquisition and systematically responds to written words in literate individuals. According to the neuronal recycling hypothesis, this region arises through the repurposing, for letter recognition, of a subpart of the ventral visual pathway initially involved in face and object recognition. Furthermore, according to the biased connectivity hypothesis, its reproducible localization is due to preexisting connections from this subregion to areas involved in spoken-language processing. Here, we evaluate those hypotheses in an explicit computational model. We trained a deep convolutional neural network of the ventral visual pathway, first to categorize pictures and then to recognize written words invariantly for case, font, and size. We show that the model can account for many properties of the VWFA, particularly when a subset of units possesses a biased connectivity to word output units. The network develops a sparse, invariant representation of written words, based on a restricted set of reading-selective units. Their activation mimics several properties of the VWFA, and their lesioning causes a reading-specific deficit. The model predicts that, in literate brains, written words are encoded by a compositional neural code with neurons tuned either to individual letters and their ordinal position relative to word start or word ending or to pairs of letters (bigrams).

Sentiment annotations for 3827 simplified Chinese characters.

Sentiment analysis in Chinese natural language processing has been largely based on words annotated with sentiment categories or scores. Characters, however, are the basic orthographic, phonological, and in most cases, semantic units in the Chinese language. This study collected sentiment annotations for 3827 characters. The ratings demonstrated high levels of reliability, and were validated through a comparison with the ratings of some characters' word equivalents reported in a previous norming study. Relations with other lexico-semantic variables and character processing efficiency were investigated. Furthermore, analyses of the association between constituent character valence and word valence revealed semantic compositionality and sentiment fusion characteristic of larger Chinese linguistic units. These ratings for characters, expanding current Chinese sentiment lexicons, can be utilized for the purposes of more precise stimuli assessment in research on Chinese character processing and more efficient sentiment analysis equipped with annotations of single-character words.

Generative cultural learning in children and adults: the role of compositionality and generativity in cultural evolution.

Are human cultures distinctively cumulative because they are uniquely compositional? We addressed this question using a summative learning paradigm where participants saw different models build different tower elements, consisting of discrete actions and objects: stacking cubes (tower base) and linking squares (tower apex). These elements could be combined to form a tower that was optimal in terms of height and structural soundness. In addition to measuring copying fidelity, we explored whether children and adults (i) extended the knowledge demonstrated to additional tower elements and (ii) productively combined them. Results showed that children and adults copied observed demonstrations and applied them to novel exemplars. However, only adults in the imitation condition combined the two newly derived base and apex, relative to adults in a control group. Nonetheless, there were remarkable similarities between children's and adults' performance across measures. Composite measures capturing errors and overall generativity in children's and adults' performance produced few population by condition interactions. Results suggest that early in development, humans possess a suite of cognitive skills-compositionality and generativity-that transforms phylogenetically widespread social learning competencies into something that may be unique to our species, cultural learning; allowing human cultures to evolve towards greater complexity.

Network Analysis of Human Brain Connectivity Reveals Neural Fingerprints of a Compositionality Bias in Signaling Systems.

Compositionality is a hallmark of human language and other symbolic systems: a finite set of meaningful elements can be systematically combined to convey an open-ended array of ideas. Compositionality is not uniformly distributed over expressions in a language or over individuals' communicative behavior: at both levels, variation is observed. Here, we investigate the neural bases of interindividual variability by probing the relationship between intrinsic characteristics of brain networks and compositional behavior. We first collected functional resting-state and diffusion magnetic resonance imaging data from a large participant sample (N = 51). Subsequently, participants took part in two signaling games. They were instructed to learn and reproduce an auditory symbolic system of signals (tone sequences) associated with affective meanings (human faces expressing emotions). Signal-meaning mappings were artificial and had to be learned via repeated signaling interactions. We identified a temporoparietal network in which connection length was related to the degree of compositionality introduced in a signaling system by each player. Graph-theoretic analysis of resting-state functional connectivity revealed that, within that network, compositional behavior was associated with integration measures in 2 semantic hubs: the left posterior cingulate cortex and the left angular gyrus. Our findings link individual variability in compositional biases to variation in the anatomy of semantic networks and in the functional topology of their constituent units.

The role of left angular gyrus in the representation of linguistic composition relations.

Language comprehension is compositional: individual words are combined structurally to form larger meaning representations. The neural basis for compositionality is at the center of a growing body of recent research. Previous work has largely used univariate analysis to investigate the question, a technique that could potentially lead to the loss of fined-grained information due to the procedure of averaging over neural responses. In a functional magnetic resonance imaging experiment, the present study examined different types of composition relations in Chinese phrases, using a 1-back composition relation probe (CRP) task and a 1-back word probe (WP) task. We first analyzed the data using the multivariate representation similarity analysis, which better captures the fine-grained representational differences in the stimuli. The results showed that the left angular gyrus (AG) represents different types of composition relations in the CRP task, but no brain areas were identified in the WP task. We also conducted a traditional univariate analysis and found greater activations in the bilateral inferior frontal gyrus in the CRP task relative to the WP task. We discuss the methodological and theoretical implications of our findings in the context of the larger language neural network identified in previous studies. Our findings highlight the role of left AG in representing and distinguishing fine-grained linguistic composition relations.

Overlooked evidence for semantic compositionality and signal reduction in wild chimpanzees (Pan troglodytes).

Recent discoveries of semantic compositionality in Japanese tits have enlivened the discussions on the presence of this phenomenon in wild animal communication. Data on semantic compositionality in wild apes are lacking, even though language experiments with captive apes have demonstrated they are capable of semantic compositionality. In this paper, I revisit the study by Boesch (Hum. Evol. 6:81-89, 1991) who investigated drumming sequences by an alpha male in a chimpanzee (Pan troglodytes) community in the Taï National Park, Côte d'Ivoire. A reanalysis of the data reveals that the alpha male produced semantically compositional combined messages of travel direction change and resting period initiation. Unlike the Japanese tits, the elements of the compositional expression were not simply juxtaposed but displayed structural reduction, while one of the two elements in the expression coded the meanings of both elements. These processes show relative resemblance to blending and fusion in human languages. Also unlike the tits, the elements of the compositional expression did not have a fixed order, although there was a fixed distribution of drumming events across the trees used for drumming. Because the elements of the expression appear to carry verb-like meanings, the compositional expression also resembles simple verb-verb constructions and short paratactic combinations of two clauses found across languages. In conclusion, the reanalysis suggests that semantic compositionality and phenomena resembling paratactic combinations of two clauses might have been present in the communication of the last common ancestor of chimpanzees and humans, not necessarily in the vocal modality.

A language of thought for the mental representation of geometric shapes.

In various cultures and at all spatial scales, humans produce a rich complexity of geometric shapes such as lines, circles or spirals. Here, we propose that humans possess a language of thought for geometric shapes that can produce line drawings as recursive combinations of a minimal set of geometric primitives. We present a programming language, similar to Logo, that combines discrete numbers and continuous integration to form higher-level structures based on repetition, concatenation and embedding, and we show that the simplest programs in this language generate the fundamental geometric shapes observed in human cultures. On the perceptual side, we propose that shape perception in humans involves searching for the shortest program that correctly draws the image (program induction). A consequence of this framework is that the mental difficulty of remembering a shape should depend on its minimum description length (MDL) in the proposed language. In two experiments, we show that encoding and processing of geometric shapes is well predicted by MDL. Furthermore, our hypotheses predict additive laws for the psychological complexity of repeated, concatenated or embedded shapes, which we confirm experimentally.

The Compositionality of Facial Expressions.

The principle of compositionality, an important postulation in language and cognition research, posits that the meaning of a complex expression is determined by the meaning of its constituting parts and the operation performed on those parts. Here, we provide strong evidence that this principle plays a significant role also in interpreting facial expressions. In three studies in which perceivers interpreted sequences of two emotional facial expression images, we show that the composite meaning of facial expressions results from the meaning of its constituting expressions and an algebraic operation performed on those expressions. Our study offers a systematic account as to how the meaning of facial expressions (single and sequences) are being formed and perceived. In a broader context, our results raise the possibility that the principle of compositionality may apply to human communication modalities beyond spoken language, whereby a minimal number of components are expanded to a much greater number of meanings.

Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji.

Compositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these "upfixes" (i.e., upwards affixes) involve a productive schema enabling both stored and novel face-upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.

Compositionality of the Constituent Characters in Chinese Two-Character-Word Recognition by Adult Readers of High and Low Chinese Proficiency.

In Chinese, the graphic units are Chinese characters, most of which are compound characters. Since a compound character can be different from another one in being regarded as composed of components (compositionality), readers might have developed a compositionality awareness of the constituent characters in two-character word (2C-word) recognition. Two experiments were conducted in a lexical decision task on the same set of 2C-words, the first constituent characters of which were manipulated in compositionality. Given that a Chinese character is more difficult to recognize when it is presented upside-down than when it is presented in an upright orientation and that it is inevitable to perceive the constituent characters in 2C-word recognition, we manipulated the first constituent characters' presentation orientation to increase the task difficulty. The two constituent characters of a 2C-word target were displayed simultaneously in a trial in Experiment 1 but were shown sequentially in Experiment 2. Participants were two cohorts of adult Chinese native speakers (CNS1s and CNS2s). CNS1s had a significantly lower level of reading proficiency than CNS2s. The influence of orientation was observed in both CNS1s and CNS2s' performance across the two experiments, but only CNS2s' reaction times seemed to have indicated the effect of compositionality in Experiment 2. Skilled readers are more likely than less skilled readers to be conscious of compositionality of the first constituent characters, which are presented separately from the second ones, in 2C-word recognition.

Compositional Sequence Generation in the Entorhinal-Hippocampal System.

Neurons in the medial entorhinal cortex exhibit multiple, periodically organized, firing fields which collectively appear to form an internal representation of space. Neuroimaging data suggest that this grid coding is also present in other cortical areas such as the prefrontal cortex, indicating that it may be a general principle of neural functionality in the brain. In a recent analysis through the lens of dynamical systems theory, we showed how grid coding can lead to the generation of a diversity of empirically observed sequential reactivations of hippocampal place cells corresponding to traversals of cognitive maps. Here, we extend this sequence generation model by describing how the synthesis of multiple dynamical systems can support compositional cognitive computations. To empirically validate the model, we simulate two experiments demonstrating compositionality in space or in time during sequence generation. Finally, we describe several neural network architectures supporting various types of compositionality based on grid coding and highlight connections to recent work in machine learning leveraging analogous techniques.

Two Ways to Build a Thought: Distinct Forms of Compositional Semantic Representation across Brain Regions.

To understand a simple sentence such as "the woman chased the dog", the human mind must dynamically organize the relevant concepts to represent who did what to whom. This structured recombination of concepts (woman, dog, chased) enables the representation of novel events, and is thus a central feature of intelligence. Here, we use functional magnetic resonance (fMRI) and encoding models to delineate the contributions of three brain regions to the representation of relational combinations. We identify a region of anterior-medial prefrontal cortex (amPFC) that shares representations of noun-verb conjunctions across sentences: for example, a combination of "woman" and "chased" to encode woman-as-chaser, distinct from woman-as-chasee. This PFC region differs from the left-mid superior temporal cortex (lmSTC) and hippocampus, two regions previously implicated in representing relations. lmSTC represents broad role combinations that are shared across verbs (e.g., woman-as-agent), rather than narrow roles, limited to specific actions (woman-as-chaser). By contrast, a hippocampal sub-region represents events sharing narrow conjunctions as dissimilar. The success of the hippocampal conjunctive encoding model is anti-correlated with generalization performance in amPFC on a trial-by-trial basis, consistent with a pattern separation mechanism. Thus, these three regions appear to play distinct, but complementary, roles in encoding compositional event structure.

Concepts and Compositionality: In Search of the Brain's Language of Thought.

Imagine Genghis Khan, Aretha Franklin, and the Cleveland Cavaliers performing an opera on Maui. This silly sentence makes a serious point: As humans, we can flexibly generate and comprehend an unbounded number of complex ideas. Little is known, however, about how our brains accomplish this. Here we assemble clues from disparate areas of cognitive neuroscience, integrating recent research on language, memory, episodic simulation, and computational models of high-level cognition. Our review is framed by Fodor's classic language of thought hypothesis, according to which our minds employ an amodal, language-like system for combining and recombining simple concepts to form more complex thoughts. Here, we highlight emerging work on combinatorial processes in the brain and consider this work's relation to the language of thought. We review evidence for distinct, but complementary, contributions of map-like representations in subregions of the default mode network and sentence-like representations of conceptual relations in regions of the temporal and prefrontal cortex.

Reasoning about conscious experience with axiomatic and graphical mathematics.

We cast aspects of consciousness in axiomatic mathematical terms, using the graphical calculus of general process theories (a.k.a symmetric monoidal categories and Frobenius algebras therein). This calculus exploits the ontological neutrality of process theories. A toy example using the axiomatic calculus is given to show the power of this approach, recovering other aspects of conscious experience, such as external and internal subjective distinction, privacy or unreadability of personal subjective experience, and phenomenal unity, one of the main issues for scientific studies of consciousness. In fact, these features naturally arise from the compositional nature of axiomatic calculus.

A Compositional Model of Consciousness Based on Consciousness-Only.

Scientific studies of consciousness rely on objects whose existence is assumed to be independent of any consciousness. On the contrary, we assume consciousness to be fundamental, and that one of the main features of consciousness is characterized as being other-dependent. We set up a framework which naturally subsumes this feature by defining a compact closed category where morphisms represent conscious processes. These morphisms are a composition of a set of generators, each being specified by their relations with other generators, and therefore co-dependent. The framework is general enough and fits well into a compositional model of consciousness. Interestingly, we also show how our proposal may become a step towards avoiding the hard problem of consciousness, and thereby address the combination problem of conscious experiences.

From motor planning to execution: a sensorimotor loop perspective.

How is an evanescent wish to move translated into a concrete action? This simple question and puzzling miracle remains a focal point of motor systems neuroscience. Where does the difficulty lie? A great deal has been known about biomechanics for quite some time. More recently, there have been significant advances in our understanding of how the spinal system is organized into modules corresponding to spinal synergies, which are fixed patterns of multimuscle recruitment. But much less is known about how the supraspinal system recruits these synergies in the correct spatiotemporal pattern to effectively control movement. We argue that what makes the problem of supraspinal control so difficult is that it emerges as a result of multiple convergent and redundant sensorimotor loops. Because these loops are convergent, multiple modes of information are mixed before being sent to the spinal system; because they are redundant, information is overlapping such that a mechanism must exist to eliminate the redundancy before the signal is sent to the spinal system. Given these complex interactions, simple correlation analyses between movement variables and neural activity are likely to render a confusing and inconsistent picture. Here, we suggest that the perspective of sensorimotor loops might help in achieving a better systems-level understanding. Furthermore, state-of-the-art techniques in neurotechnology, such as optogenetics, appear to be well suited for investigating the problem of motor control at the level of loops.