Artificial intelligence detects awareness of functional relation with the environment in 3 month old babies.

A recent experiment probed how purposeful action emerges in early life by manipulating infants' functional connection to an object in the environment (i.e., tethering an infant's foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism~world connection.

Solving large-scale MEG/EEG source localisation and functional connectivity problems simultaneously using state-space models.

State-space models are widely employed across various research disciplines to study unobserved dynamics. Conventional estimation techniques, such as Kalman filtering and expectation maximisation, offer valuable insights but incur high computational costs in large-scale analyses. Sparse inverse covariance estimators can mitigate these costs, but at the expense of a trade-off between enforced sparsity and increased estimation bias, necessitating careful assessment in low signal-to-noise ratio (SNR) situations. To address these challenges, we propose a three-fold solution: (1) Introducing multiple penalised state-space (MPSS) models that leverage data-driven regularisation; (2) Developing novel algorithms derived from backpropagation, gradient descent, and alternating least squares to solve MPSS models; (3) Presenting a K-fold cross-validation extension for evaluating regularisation parameters. We validate this MPSS regularisation framework through lower and more complex simulations under varying SNR conditions, including a large-scale synthetic magneto- and electro-encephalography (MEG/EEG) data analysis. In addition, we apply MPSS models to concurrently solve brain source localisation and functional connectivity problems for real event-related MEG/EEG data, encompassing thousands of sources on the cortical surface. The proposed methodology overcomes the limitations of existing approaches, such as constraints to small-scale and region-of-interest analyses. Thus, it may enable a more accurate and detailed exploration of cognitive brain functions.

Meaning from movement and stillness: Signatures of coordination dynamics reveal infant agency.

How do human beings make sense of their relation to the world and realize their ability to effect change? Applying modern concepts and methods of coordination dynamics, we demonstrate that patterns of movement and coordination in 3 to 4-mo-olds may be used to identify states and behavioral phenotypes of emergent agency. By means of a complete coordinative analysis of baby and mobile motion and their interaction, we show that the emergence of agency can take the form of a punctuated self-organizing process, with meaning found both in movement and stillness.

2D Capsule Networks Detect Perceived Changes in Infant∼Environment Relationship Reflected in 3D Movement Dynamics.

Can infant exploration and causal discovery be detected using Artificial Intelligence (AI)? A recent experiment probed how purposeful action emerges in early life by manipulating infants' functional connection to an object in the environment (i.e., tethering one foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism∼world connection. Pairing theory-driven experimentation with AI tools thus opens a path to developing functionally-relevant assessments of infant behaviour that are likely to be useful in clinical settings.

The Haken-Kelso-Bunz (HKB) model: from matter to movement to mind.

This article presents a brief retrospective on the Haken-Kelso-Bunz (HKB) model of certain dynamical properties of human movement. Though unanticipated, HKB introduced, and demonstrated the power of, a new vocabulary for understanding behavior, cognition and the brain, revealed through a visually compelling mathematical picture that accommodated highly reproducible experimental facts and predicted new ones. HKB stands as a harbinger of paradigm change in several scientific fields, the effects of which are still being felt. In particular, HKB constitutes the foundation of a mechanistic science of coordination called Coordination Dynamics that extends from matter to movement to mind, and beyond.

Unifying Large- and Small-Scale Theories of Coordination.

Coordination is a ubiquitous feature of all living things. It occurs by virtue of informational coupling among component parts and processes and can be quite specific (as when cells in the brain resonate to signals in the environment) or nonspecific (as when simple diffusion creates a source-sink dynamic for gene networks). Existing theoretical models of coordination-from bacteria to brains to social groups-typically focus on systems with very large numbers of elements (N→∞) or systems with only a few elements coupled together (typically N = 2). Though sharing a common inspiration in Nature's propensity to generate dynamic patterns, both approaches have proceeded largely independent of each other. Ideally, one would like a theory that applies to phenomena observed on all scales. Recent experimental research by Mengsen Zhang and colleagues on intermediate-sized ensembles (in between the few and the many) proves to be the key to uniting large- and small-scale theories of coordination. Disorder-order transitions, multistability, order-order phase transitions, and especially metastability are shown to figure prominently on multiple levels of description, suggestive of a basic Coordination Dynamics that operates on all scales. This unified coordination dynamics turns out to be a marriage of two well-known models of large- and small-scale coordination: the former based on statistical mechanics (Kuramoto) and the latter based on the concepts of Synergetics and nonlinear dynamics (extended Haken-Kelso-Bunz or HKB). We show that models of the many and the few, previously quite unconnected, are thereby unified in a single formulation. The research has led to novel topological methods to handle the higher-dimensional dynamics of coordination in complex systems and has implications not only for understanding coordination but also for the design of (biorhythm inspired) computers.

Interactive Psychometrics for Autism With the Human Dynamic Clamp: Interpersonal Synchrony From Sensorimotor to Sociocognitive Domains.

The human dynamic clamp (HDC) is a human-machine interface designed on the basis of coordination dynamics for studying realistic social interaction under controlled and reproducible conditions. Here, we propose to probe the validity of the HDC as a psychometric instrument for quantifying social abilities in children with autism spectrum disorder (ASD) and neurotypical development. To study interpersonal synchrony with the HDC, we derived five standardized scores following a gradient from sensorimotor and motor to higher sociocognitive skills in a sample of 155 individuals (113 participants with ASD, 42 typically developing participants; aged 5 to 25 years; IQ > 70). Regression analyses were performed using normative modeling on global scores according to four subconditions (HDC behavior "cooperative/competitive," human task "in-phase/anti-phase," diagnosis, and age at inclusion). Children with ASD had lower scores than controls for motor skills. HDC motor coordination scores were the best candidates for stratification and diagnostic biomarkers according to exploratory analyses of hierarchical clustering and multivariate classification. Independently of phenotype, sociocognitive skills increased with developmental age while being affected by the ongoing task and HDC behavior. Weaker performance in ASD for motor skills suggests the convergent validity of the HDC for evaluating social interaction. Results provided additional evidence of a relationship between sensorimotor and sociocognitive skills. HDC may also be used as a marker of maturation of sociocognitive skills during real-time social interaction. Through its standardized and objective evaluation, the HDC not only represents a valid paradigm for the study of interpersonal synchrony but also offers a promising, clinically relevant psychometric instrument for the evaluation and stratification of sociomotor dysfunctions.

Degeneracy and Complexity in Neuro-Behavioral Correlates of Team Coordination.

Team coordination-members of a group acting together rather than performing specific actions individually-is essential for success in many real-world tasks such as military missions, sports, workplace, or school interactions. However, team coordination is highly variable, which is one reason why its underlying neural processes are largely unknown. Here we used dual electroencephalography (EEG) in dyads to study the neurobehavioral dynamics of team coordination in an ecologically valid task that places intensive demands on joint performance. We present a novel conceptual framework to interpret neurobehavioral variability in terms of degeneracy, a fundamental property of complex biological systems said to enhance flexibility and robustness. We characterize degeneracy conceptually in terms of a manifold representing the geometric locus of the dynamics in the high dimensional state-space of neurobehavioral signals. The geometry and dimensionality of the manifold are determined by task constraints and team coordination requirements which restrict the manifold to trajectories that are conducive to successful task performance. Our results indicate that team coordination is associated with dimensionality reduction of the manifold as evident in increased inter-brain phase coherence of beta and gamma rhythms during critical phases of task performance where subjects exchange information. Team coordination was also found to affect the shape of the manifold manifested as a symmetry breaking of centro-parietal wavelet power patterns across subjects in trials with high team coordination. These results open a conceptual and empirical path to identifying the mechanisms underlying team performance in complex tasks.

Coordination Dynamics: A Foundation for Understanding Social Behavior.

Humans' interactions with each other or with socially competent machines exhibit lawful coordination patterns at multiple levels of description. According to Coordination Dynamics, such laws specify the flow of coordination states produced by functional synergies of elements (e.g., cells, body parts, brain areas, people…) that are temporarily organized as single, coherent units. These coordinative structures or synergies may be mathematically characterized as informationally coupled self-organizing dynamical systems (Coordination Dynamics). In this paper, we start from a simple foundation, an elemental model system for social interactions, whose behavior has been captured in the Haken-Kelso-Bunz (HKB) model. We follow a tried and tested scientific method that tightly interweaves experimental neurobehavioral studies and mathematical models. We use this method to further develop a body of empirical research that advances the theory toward more generalized forms. In concordance with this interdisciplinary spirit, the present paper is written both as an overview of relevant advances and as an introduction to its mathematical underpinnings. We demonstrate HKB's evolution in the context of social coordination along several directions, with its applicability growing to increasingly complex scenarios. In particular, we show that accommodating for symmetry breaking in intrinsic dynamics and coupling, multiscale generalization and adaptation are principal evolutions. We conclude that a general framework for social coordination dynamics is on the horizon, in which models support experiments with hypothesis generation and mechanistic insights.

Metastable neural dynamics underlies cognitive performance across multiple behavioural paradigms.

Despite resting state networks being associated with a variety of cognitive abilities, it remains unclear how these local areas act in concert to express particular cognitive operations. Theoretical and empirical accounts indicate that large-scale resting state networks reconcile dual tendencies towards integration and segregation by operating in a metastable regime of their coordination dynamics. Metastability may confer important behavioural qualities by binding distributed local areas into large-scale neurocognitive networks. We tested this hypothesis by analysing fMRI data in a large cohort of healthy individuals (N = 566) and comparing the metastability of the brain's large-scale resting network architecture at rest and during the performance of several tasks. Metastability was estimated using a well-defined collective variable capturing the level of 'phase-locking' between large-scale networks over time. Task-based reasoning was principally characterised by high metastability in cognitive control networks and low metastability in sensory processing areas. Although metastability between resting state networks increased during task performance, cognitive ability was more closely linked to spontaneous activity. High metastability in the intrinsic connectivity of cognitive control networks was linked to novel problem solving or fluid intelligence, but was less important in tasks relying on previous experience or crystallised intelligence. Crucially, subjects with resting architectures similar or 'pre-configured' to a task-general arrangement demonstrated superior cognitive performance. Taken together, our findings support a key linkage between the spontaneous metastability of large-scale networks in the cerebral cortex and cognition.

Topological portraits of multiscale coordination dynamics.

Living systems exhibit complex yet organized behavior on multiple spatiotemporal scales. To investigate the nature of multiscale coordination in living systems, one needs a meaningful and systematic way to quantify the complex dynamics, a challenge in both theoretical and empirical realms. The present work shows how integrating approaches from computational algebraic topology and dynamical systems may help us meet this challenge. In particular, we focus on the application of multiscale topological analysis to coordinated rhythmic processes. First, theoretical arguments are introduced as to why certain topological features and their scale-dependency are highly relevant to understanding complex collective dynamics. Second, we propose a method to capture such dynamically relevant topological information using persistent homology, which allows us to effectively construct a multiscale topological portrait of rhythmic coordination. Finally, the method is put to test in detecting transitions in real data from an experiment of rhythmic coordination in ensembles of interacting humans. The recurrence plots of topological portraits highlight collective transitions in coordination patterns that were elusive to more traditional methods. This sensitivity to collective transitions would be lost if the behavioral dynamics of individuals were treated as separate degrees of freedom instead of constituents of the topology that they collectively forge. Such multiscale topological portraits highlight collective aspects of coordination patterns that are irreducible to properties of individual parts. The present work demonstrates how the analysis of multiscale coordination dynamics can benefit from topological methods, thereby paving the way for further systematic quantification of complex, high-dimensional dynamics in living systems.

Spectral dissociation of lateralized brain rhythms.

Using high resolution spectral methods to uncover neuromarkers of social, cognitive and behavioral function, we have found that hemi-lateralized pairs of oscillations such as left and right occipital alpha or left and right rolandic mu dissociate spectrally. That is, they show a shifted frequency distribution, with one member of the pair peaking at a slightly lower frequency than the other. Resorting to the analysis of EEG spatio-spectral patterns, we provide examples of dissociations in the 10Hz frequency band. Occasionally, hemi-lateralized pairs blend into medial aggregates, probably when functional interactions lead to strongly coherent dynamics through frequency-locking or metastability. Our observations support the hypothesis that homologous pairs of neuromarkers have characteristically distinct intrinsic frequencies and coordinate their oscillations into synchronous ensembles only transiently. This property could play a role in the balance of integration and segregation in the brain: spectral separation of the oscillations from homologous cortical areas allows them to function independently under certain circumstances, all the while preserving a potential for stronger interactions supported by structural and functional symmetries. Spectral dissociation (and its methodological corollary: spectral analysis with high frequency resolution) may be harnessed to better track the individual power of each member of a hemi-lateralized pair and their respective time-course, leading to enhanced internal validity and reproducibility of research on neural oscillations. Resulting insights may shed light on the functional interaction between homologous cortices in studies of attention (alpha), e.g. during perceptual and social interaction tasks, and in studies of somatomotor processing (mu), e.g. in bimanual coordination and neuroprosthetics.

Connecting empirical phenomena and theoretical models of biological coordination across scales.

Coordination in living systems-from cells to people-must be understood at multiple levels of description. Analyses and modelling of empirically observed patterns of biological coordination often focus either on ensemble-level statistics in large-scale systems with many components, or on detailed dynamics in small-scale systems with few components. The two approaches have proceeded largely independent of each other. To bridge this gap between levels and scales, we have recently conducted a human experiment of mid-scale social coordination specifically designed to reveal coordination at multiple levels (ensemble, subgroups and dyads) simultaneously. Based on this experiment, the present work shows that, surprisingly, a single system of equations captures key observations at all relevant levels. It also connects empirically validated models of large- and small-scale biological coordination-the Kuramoto and extended Haken-Kelso-Bunz (HKB) models-and the hallmark phenomena that each is known to capture. For example, it exhibits both multistability and metastability observed in small-scale empirical research (via the second-order coupling and symmetry breaking in extended HKB) and the growth of biological complexity as a function of scale (via the scalability of the Kuramoto model). Only by incorporating both of these features simultaneously can we reproduce the essential coordination behaviour observed in our experiment.

Metastable neural dynamics in Alzheimer's disease are disrupted by lesions to the structural connectome.

Current theory suggests brain regions interact to reconcile the competing demands of integration and segregation by leveraging metastable dynamics. An emerging consensus recognises the importance of metastability in healthy neural dynamics where the transition between network states over time is dependent upon the structural connectivity between brain regions. In Alzheimer's disease (AD) - the most common form of dementia - these couplings are progressively weakened, metastability of neural dynamics are reduced and cognitive ability is impaired. Accordingly, we use a joint empirical and computational approach to reveal how behaviourally relevant changes in neural metastability are contingent on the structural integrity of the anatomical connectome. We estimate the metastability of fMRI BOLD signal in subjects from across the AD spectrum and in healthy controls and demonstrate the dissociable effects of structural disconnection on synchrony versus metastability. In addition, we reveal the critical role of metastability in general cognition by demonstrating the link between an individuals cognitive performance and their metastable neural dynamic. Finally, using whole-brain computer modelling, we demonstrate how a healthy neural dynamic is conditioned upon the topological integrity of the structural connectome. Overall, the results of our joint computational and empirical analysis suggest an important causal relationship between metastable neural dynamics, cognition, and the structural efficiency of the anatomical connectome.

Critical diversity: Divided or united states of social coordination.

Much of our knowledge of coordination comes from studies of simple, dyadic systems or systems containing large numbers of components. The huge gap 'in between' is seldom addressed, empirically or theoretically. We introduce a new paradigm to study the coordination dynamics of such intermediate-sized ensembles with the goal of identifying key mechanisms of interaction. Rhythmic coordination was studied in ensembles of eight people, with differences in movement frequency ('diversity') manipulated within the ensemble. Quantitative change in diversity led to qualitative changes in coordination, a critical value separating régimes of integration and segregation between groups. Metastable and multifrequency coordination between participants enabled communication across segregated groups within the ensemble, without destroying overall order. These novel findings reveal key factors underlying coordination in ensemble sizes previously considered too complicated or 'messy' for systematic study and supply future theoretical/computational models with new empirical checkpoints.

A roadmap to computational social neuroscience.

To complement experimental efforts toward understanding human social interactions at both neural and behavioral levels, two computational approaches are presented: (1) a fully parameterizable mathematical model of a social partner, the Human Dynamic Clamp which, by virtue of experimentally controlled interactions between Virtual Partners and real people, allows for emergent behaviors to be studied; and (2) a multiscale neurocomputational model of social coordination that enables exploration of social self-organization at all levels-from neuronal patterns to people interacting with each other. These complementary frameworks and the cross product of their analysis aim at understanding the fundamental principles governing social behavior.

How Interpersonal Coordination Affects Individual Behavior (and Vice Versa): Experimental analysis and adaptive HKB model of social memory.

How one behaves after interacting with a friend may not be the same as before the interaction. The present study investigated which spontaneous coordination patterns formed between two persons and whether a remnant of the interaction remained ("social memory"). Pairs of people sat face-to-face and continuously flexed index fingers while vision between partners was manipulated to allow or prevent information exchange. Trials consisted of three successive twenty-second intervals: without vision, with vision, and again without vision. Steady, transient, or absent phase coupling was observed during vision. In support of social memory, participants tended to remain near each other's movement frequency after the interaction ended. Furthermore, the greater the stability of interpersonal coordination, the more similar partners' post-interactional frequencies became. Proposing that social memory resulted from prior frequency adaptation, a model based on Haken-Kelso-Bunz oscillators reproduced the experimental findings, even for patterns observed on individual trials. Parametric manipulations revealed multiple routes to social memory through the interplay of adaptation and other model parameters. The experimental results, model, and interpretation motivate potential future research and therapeutic applications.

Coordination Dynamics in Cognitive Neuroscience.

Many researchers and clinicians in cognitive neuroscience hold to a modular view of cognitive function in which the cerebral cortex operates by the activation of areas with circumscribed elementary cognitive functions. Yet an ongoing paradigm shift to a dynamic network perspective is underway. This new viewpoint treats cortical function as arising from the coordination dynamics within and between cortical regions. Cortical coordination dynamics arises due to the unidirectional influences imposed on a cortical area by inputs from other areas that project to it, combined with the projection reciprocity that characterizes cortical connectivity and gives rise to reentrant processing. As a result, cortical dynamics exhibits both segregative and integrative tendencies and gives rise to both cooperative and competitive relations within and between cortical areas that are hypothesized to underlie the emergence of cognition in brains.

On the Self-Organizing Origins of Agency.

The question of agency and directedness in living systems has puzzled philosophers and scientists for centuries. What principles and mechanisms underlie the emergence of agency? Analysis and dynamical modeling of experiments on human infants suggest that the birth of agency is due to a eureka-like, pattern-forming phase transition in which the infant suddenly realizes it can make things happen in the world. The main mechanism involves positive feedback: when the baby's initially spontaneous movements cause the world to change, their perceived consequences have a sudden and sustained amplifying effect on the baby's further actions. The baby discovers itself as a causal agent. Some implications of this theory are discussed.