Correlation dimension of high-dimensional and high-definition experimental time series.

The correlation dimension (CD) is a nonlinear measure of the complexity of invariant sets. First introduced for describing low-dimensional chaotic attractors, it has been later extended to the analysis of experimental electroencephalographic (EEG), magnetoencephalographic (MEG), and local field potential (LFP) recordings. However, its direct application to high-dimensional (dozens of signals) and high-definition (kHz sampling rate) 2HD data revealed a controversy in the results. We show that the need for an exponentially long data sample is the main difficulty in dealing with 2HD data. Then, we provide a novel method for estimating CD that enables orders of magnitude reduction of the required sample size. The approach decomposes raw data into statistically independent components and estimates the CD for each of them separately. In addition, the method allows ongoing insights into the interplay between the complexity of the contributing components, which can be related to different anatomical pathways and brain regions. The latter opens new approaches to a deeper interpretation of experimental data. Finally, we illustrate the method with synthetic data and LFPs recorded in the hippocampus of a rat.

Site-dependent shaping of field potential waveforms.

The activity of neuron populations gives rise to field potentials (FPs) that extend beyond the sources. Their mixing in the volume dilutes the original temporal motifs in a site-dependent manner, a fact that has received little attention. And yet, it potentially rids of physiological significance the time-frequency parameters of individual waves (amplitude, phase, duration). This is most likely to happen when a single source or a local origin is erroneously assumed. Recent studies using spatial treatment of these signals and anatomically realistic modeling of neuron aggregates provide convincing evidence for the multisource origin and site-dependent blend of FPs. Thus, FPs generated in primary structures like the neocortex and hippocampus reach far and cross-contaminate each other but also, they add and even impose their temporal traits on distant regions. Furthermore, both structures house neurons that act as spatially distinct (but overlapped) FP sources whose activation is state, region, and time dependent, making the composition of so-called local FPs highly volatile and strongly site dependent. Since the spatial reach cannot be predicted without source geometry, it is important to assess whether waveforms and temporal motifs arise from a single source; otherwise, those from each of the co-active sources should be sought.

Toward Reflective Spiking Neural Networks Exploiting Memristive Devices.

The design of modern convolutional artificial neural networks (ANNs) composed of formal neurons copies the architecture of the visual cortex. Signals proceed through a hierarchy, where receptive fields become increasingly more complex and coding sparse. Nowadays, ANNs outperform humans in controlled pattern recognition tasks yet remain far behind in cognition. In part, it happens due to limited knowledge about the higher echelons of the brain hierarchy, where neurons actively generate predictions about what will happen next, i.e., the information processing jumps from reflex to reflection. In this study, we forecast that spiking neural networks (SNNs) can achieve the next qualitative leap. Reflective SNNs may take advantage of their intrinsic dynamics and mimic complex, not reflex-based, brain actions. They also enable a significant reduction in energy consumption. However, the training of SNNs is a challenging problem, strongly limiting their deployment. We then briefly overview new insights provided by the concept of a high-dimensional brain, which has been put forward to explain the potential power of single neurons in higher brain stations and deep SNN layers. Finally, we discuss the prospect of implementing neural networks in memristive systems. Such systems can densely pack on a chip 2D or 3D arrays of plastic synaptic contacts directly processing analog information. Thus, memristive devices are a good candidate for implementing in-memory and in-sensor computing. Then, memristive SNNs can diverge from the development of ANNs and build their niche, cognitive, or reflective computations.

Spatial Memory in a Spiking Neural Network with Robot Embodiment.

Cognitive maps and spatial memory are fundamental paradigms of brain functioning. Here, we present a spiking neural network (SNN) capable of generating an internal representation of the external environment and implementing spatial memory. The SNN initially has a non-specific architecture, which is then shaped by Hebbian-type synaptic plasticity. The network receives stimuli at specific loci, while the memory retrieval operates as a functional SNN response in the form of population bursts. The SNN function is explored through its embodiment in a robot moving in an arena with safe and dangerous zones. We propose a measure of the global network memory using the synaptic vector field approach to validate results and calculate information characteristics, including learning curves. We show that after training, the SNN can effectively control the robot's cognitive behavior, allowing it to avoid dangerous regions in the arena. However, the learning is not perfect. The robot eventually visits dangerous areas. Such behavior, also observed in animals, enables relearning in time-evolving environments. If a dangerous zone moves into another place, the SNN remaps positive and negative areas, allowing escaping the catastrophic interference phenomenon known for some AI architectures. Thus, the robot adapts to changing world.

Static internal representation of dynamic situations reveals time compaction in human cognition.

INTRODUCTION: The human brain has evolved under the constraint of survival in complex dynamic situations. It makes fast and reliable decisions based on internal representations of the environment. Whereas neural mechanisms involved in the internal representation of space are becoming known, entire spatiotemporal cognition remains a challenge. Growing experimental evidence suggests that brain mechanisms devoted to spatial cognition may also participate in spatiotemporal information processing. OBJECTIVES: The time compaction hypothesis postulates that the brain represents both static and dynamic situations as purely static maps. Such an internal reduction of the external complexity allows humans to process time-changing situations in real-time efficiently. According to time compaction, there may be a deep inner similarity between the representation of conventional static and dynamic visual stimuli. Here, we test the hypothesis and report the first experimental evidence of time compaction in humans. METHODS: We engaged human subjects in a discrimination-learning task consisting in the classification of static and dynamic visual stimuli. When there was a hidden correspondence between static and dynamic stimuli due to time compaction, the learning performance was expected to be modulated. We studied such a modulation experimentally and by a computational model. RESULTS: The collected data validated the predicted learning modulation and confirmed that time compaction is a salient cognitive strategy adopted by the human brain to process time-changing situations. Mathematical modelling supported the finding. We also revealed that men are more prone to exploit time compaction in accordance with the context of the hypothesis as a cognitive basis for survival. CONCLUSIONS: The static internal representation of dynamic situations is a human cognitive mechanism involved in decision-making and strategy planning to cope with time-changing environments. The finding opens a new venue to understand how humans efficiently interact with our dynamic world and thrive in nature.

Universal principles justify the existence of concept cells.

The widespread consensus argues that the emergence of abstract concepts in the human brain, such as a "table", requires complex, perfectly orchestrated interaction of myriads of neurons. However, this is not what converging experimental evidence suggests. Single neurons, the so-called concept cells (CCs), may be responsible for complex tasks performed by humans. This finding, with deep implications for neuroscience and theory of neural networks, has no solid theoretical grounds so far. Our recent advances in stochastic separability of highdimensional data have provided the basis to validate the existence of CCs. Here, starting from a few first principles, we layout biophysical foundations showing that CCs are not only possible but highly likely in brain structures such as the hippocampus. Three fundamental conditions, fulfilled by the human brain, ensure high cognitive functionality of single cells: a hierarchical feedforward organization of large laminar neuronal strata, a suprathreshold number of synaptic entries to principal neurons in the strata, and a magnitude of synaptic plasticity adequate for each neuronal stratum. We illustrate the approach on a simple example of acquiring "musical memory" and show how the concept of musical notes can emerge.

Semantic Knowledge Representation for Strategic Interactions in Dynamic Situations.

Evolved living beings can anticipate the consequences of their actions in complex multilevel dynamic situations. This ability relies on abstracting the meaning of an action. The underlying brain mechanisms of such semantic processing of information are poorly understood. Here we show how our novel concept, known as time compaction, provides a natural way of representing semantic knowledge of actions in time-changing situations. As a testbed, we model a fencing scenario with a subject deciding between attack and defense strategies. The semantic content of each action in terms of lethality, versatility, and imminence is then structured as a spatial (static) map representing a particular fencing (dynamic) situation. The model allows deploying a variety of cognitive strategies in a fast and reliable way. We validate the approach in virtual reality and by using a real humanoid robot.

Spatial Properties of STDP in a Self-Learning Spiking Neural Network Enable Controlling a Mobile Robot.

Development of spiking neural networks (SNNs) controlling mobile robots is one of the modern challenges in computational neuroscience and artificial intelligence. Such networks, being replicas of biological ones, are expected to have a higher computational potential than traditional artificial neural networks (ANNs). The critical problem is in the design of robust learning algorithms aimed at building a "living computer" based on SNNs. Here, we propose a simple SNN equipped with a Hebbian rule in the form of spike-timing-dependent plasticity (STDP). The SNN implements associative learning by exploiting the spatial properties of STDP. We show that a LEGO robot controlled by the SNN can exhibit classical and operant conditioning. Competition of spike-conducting pathways in the SNN plays a fundamental role in establishing associations of neural connections. It replaces the irrelevant associations by new ones in response to a change in stimuli. Thus, the robot gets the ability to relearn when the environment changes. The proposed SNN and the stimulation protocol can be further enhanced and tested in developing neuronal cultures, and also admit the use of memristive devices for hardware implementation.

Local and Volume-Conducted Contributions to Cortical Field Potentials.

Brain field potentials (FPs) can reach far from their sources, making difficult to know which waves come from where. We show that modern algorithms efficiently segregate the local and remote contributions to cortical FPs by recovering the generator-specific spatial voltage profiles. We investigated experimentally and numerically the local and remote origin of FPs in different cortical areas in anesthetized rats. All cortices examined show significant state, layer, and region dependent contribution of remote activity, while the voltage profiles help identify their subcortical or remote cortical origin. Co-activation of different cortical modules can be discriminated by the distinctive spatial features of the corresponding profiles. All frequency bands contain remote activity, thus influencing the FP time course, in cases drastically. The reach of different FP patterns is boosted by spatial coherence and curved geometry of the sources. For instance, slow cortical oscillations reached the entire brain, while hippocampal theta reached only some portions of the cortex. In anterior cortices, most alpha oscillations have a remote origin, while in the visual cortex the remote theta and gamma even surpass the local contribution. The quantitative approach to local and distant FP contributions helps to refine functional connectivity among cortical regions, and their relation to behavior.

High-Dimensional Brain: A Tool for Encoding and Rapid Learning of Memories by Single Neurons.

Codifying memories is one of the fundamental problems of modern Neuroscience. The functional mechanisms behind this phenomenon remain largely unknown. Experimental evidence suggests that some of the memory functions are performed by stratified brain structures such as the hippocampus. In this particular case, single neurons in the CA1 region receive a highly multidimensional input from the CA3 area, which is a hub for information processing. We thus assess the implication of the abundance of neuronal signalling routes converging onto single cells on the information processing. We show that single neurons can selectively detect and learn arbitrary information items, given that they operate in high dimensions. The argument is based on stochastic separation theorems and the concentration of measure phenomena. We demonstrate that a simple enough functional neuronal model is capable of explaining: (i) the extreme selectivity of single neurons to the information content, (ii) simultaneous separation of several uncorrelated stimuli or informational items from a large set, and (iii) dynamic learning of new items by associating them with already "known" ones. These results constitute a basis for organization of complex memories in ensembles of single neurons. Moreover, they show that no a priori assumptions on the structural organization of neuronal ensembles are necessary for explaining basic concepts of static and dynamic memories.

The unreasonable effectiveness of small neural ensembles in high-dimensional brain.

Complexity is an indisputable, well-known, and broadly accepted feature of the brain. Despite the apparently obvious and widely-spread consensus on the brain complexity, sprouts of the single neuron revolution emerged in neuroscience in the 1970s. They brought many unexpected discoveries, including grandmother or concept cells and sparse coding of information in the brain. In machine learning for a long time, the famous curse of dimensionality seemed to be an unsolvable problem. Nevertheless, the idea of the blessing of dimensionality becomes gradually more and more popular. Ensembles of non-interacting or weakly interacting simple units prove to be an effective tool for solving essentially multidimensional and apparently incomprehensible problems. This approach is especially useful for one-shot (non-iterative) correction of errors in large legacy artificial intelligence systems and when the complete re-training is impossible or too expensive. These simplicity revolutions in the era of complexity have deep fundamental reasons grounded in geometry of multidimensional data spaces. To explore and understand these reasons we revisit the background ideas of statistical physics. In the course of the 20th century they were developed into the concentration of measure theory. The Gibbs equivalence of ensembles with further generalizations shows that the data in high-dimensional spaces are concentrated near shells of smaller dimension. New stochastic separation theorems reveal the fine structure of the data clouds. We review and analyse biological, physical, and mathematical problems at the core of the fundamental question: how can high-dimensional brain organise reliable and fast learning in high-dimensional world of data by simple tools? To meet this challenge, we outline and setup a framework based on statistical physics of data. Two critical applications are reviewed to exemplify the approach: one-shot correction of errors in intellectual systems and emergence of static and associative memories in ensembles of single neurons. Error correctors should be simple; not damage the existing skills of the system; allow fast non-iterative learning and correction of new mistakes without destroying the previous fixes. All these demands can be satisfied by new tools based on the concentration of measure phenomena and stochastic separation theory. We show how a simple enough functional neuronal model is capable of explaining: i) the extreme selectivity of single neurons to the information content of high-dimensional data, ii) simultaneous separation of several uncorrelated informational items from a large set of stimuli, and iii) dynamic learning of new items by associating them with already "known" ones. These results constitute a basis for organisation of complex memories in ensembles of single neurons.

Fast social-like learning of complex behaviors based on motor motifs.

Social learning is widely observed in many species. Less experienced agents copy successful behaviors exhibited by more experienced individuals. Nevertheless, the dynamical mechanisms behind this process remain largely unknown. Here we assume that a complex behavior can be decomposed into a sequence of n motor motifs. Then a neural network capable of activating motor motifs in a given sequence can drive an agent. To account for (n-1)! possible sequences of motifs in a neural network, we employ the winnerless competition approach. We then consider a teacher-learner situation: one agent exhibits a complex movement, while another one aims at mimicking the teacher's behavior. Despite the huge variety of possible motif sequences we show that the learner, equipped with the provided learning model, can rewire "on the fly" its synaptic couplings in no more than (n-1) learning cycles and converge exponentially to the durations of the teacher's motifs. We validate the learning model on mobile robots. Experimental results show that the learner is indeed capable of copying the teacher's behavior composed of six motor motifs in a few learning cycles. The reported mechanism of learning is general and can be used for replicating different functions, including, for example, sound patterns or speech.

Latent Factors Limiting the Performance of sEMG-Interfaces.

Recent advances in recording and real-time analysis of surface electromyographic signals (sEMG) have fostered the use of sEMG human-machine interfaces for controlling personal computers, prostheses of upper limbs, and exoskeletons among others. Despite a relatively high mean performance, sEMG-interfaces still exhibit strong variance in the fidelity of gesture recognition among different users. Here, we systematically study the latent factors determining the performance of sEMG-interfaces in synthetic tests and in an arcade game. We show that the degree of muscle cooperation and the amount of the body fatty tissue are the decisive factors in synthetic tests. Our data suggest that these factors can only be adjusted by long-term training, which promotes fine-tuning of low-level neural circuits driving the muscles. Short-term training has no effect on synthetic tests, but significantly increases the game scoring. This implies that it works at a higher decision-making level, not relevant for synthetic gestures. We propose a procedure that enables quantification of the gestures' fidelity in a dynamic gaming environment. For each individual subject, the approach allows identifying "problematic" gestures that decrease gaming performance. This information can be used for optimizing the training strategy and for adapting the signal processing algorithms to individual users, which could be a way for a qualitative leap in the development of future sEMG-interfaces.

The right hippocampus leads the bilateral integration of gamma-parsed lateralized information.

It is unclear whether the two hippocampal lobes convey similar or different activities and how they cooperate. Spatial discrimination of electric fields in anesthetized rats allowed us to compare the pathway-specific field potentials corresponding to the gamma-paced CA3 output (CA1 Schaffer potentials) and CA3 somatic inhibition within and between sides. Bilateral excitatory Schaffer gamma waves are generally larger and lead from the right hemisphere with only moderate covariation of amplitude, and drive CA1 pyramidal units more strongly than unilateral waves. CA3 waves lock to the ipsilateral Schaffer potentials, although bilateral coherence was weak. Notably, Schaffer activity may run laterally, as seen after the disruption of the connecting pathways. Thus, asymmetric operations promote the entrainment of CA3-autonomous gamma oscillators bilaterally, synchronizing lateralized gamma strings to converge optimally on CA1 targets. The findings support the view that interhippocampal connections integrate different aspects of information that flow through the left and right lobes.

Diversity of LFPs Activated in Different Target Regions by a Common CA3 Input.

Identifying the pathways contributing to local field potential (LFP) events and oscillations is essential to determine whether synchronous interregional patterns indicate functional connectivity. Here, we studied experimentally and numerically how different target structures receiving input from a common population shape their LFPs. We focused on the bilateral CA3 that sends gamma-paced excitatory packages to the bilateral CA1, the lateral septum, and itself (recurrent input). The CA3-specific contribution was isolated from multisite LFPs in target regions using spatial discrimination techniques. We found strong modulation of LFPs by target-specific features, including the morphology and population arrangement of cells, the timing of CA3 inputs, volume conduction from nearby targets, and co-activated inhibition. Jointly they greatly affect the LFP amplitude, profile, and frequency characteristics. For instance, ipsilateral (Schaffer) LFPs occluded contralateral ones, and septal LFPs arise mostly from remote sources while local contribution from CA3 input was minor. In the CA3 itself, gamma waves have dual origin from local networks: in-phase excitatory and nearly antiphase inhibitory. Also, waves may have different duration and varying phase in different targets. These results indicate that to explore the cellular basis of LFPs and the functional connectivity between structures, besides identifying the origin population/s, target modifiers should be considered.

Prediction-for-CompAction: navigation in social environments using generalized cognitive maps.

The ultimate navigation efficiency of mobile robots in human environments will depend on how we will appraise them: merely as impersonal machines or as human-like agents. In the latter case, an agent may take advantage of the cooperative collision avoidance, given that it possesses recursive cognition, i.e., the agent's decisions depend on the decisions made by humans that in turn depend on the agent's decisions. To deal with this high-level cognitive skill, we propose a neural network architecture implementing Prediction-for-CompAction paradigm. The network predicts possible human-agent collisions and compacts the time dimension by projecting a given dynamic situation into a static map. Thereby emerging compact cognitive map can be readily used as a "dynamic GPS" for planning actions or mental evaluation of the convenience of cooperation in a given context. We provide numerical evidence that cooperation yields additional room for more efficient navigation in cluttered pedestrian flows, and the agent can choose path to the target significantly shorter than a robot treated by humans as a functional machine. Moreover, the navigation safety, i.e., the chances to avoid accidental collisions, increases under cooperation. Remarkably, these benefits yield no additional load to the mean society effort. Thus, the proposed strategy is socially compliant, and the humanoid agent can behave as "one of us."

Can pathway-specific LFPs be obtained in cytoarchitectonically complex structures?

Deciphering how the brain encodes the continuous flow of information contained in natural stimuli requires understanding the spontaneous activity of functional assemblies in multiple neuronal populations. A promising integrative approach that combines multisite recordings of local field potentials (LFP) with an independent component analysis (ICA) enables continuous readouts of population specific activities of functionally different neuron groups to be obtained. We previously used this technique successfully in the hippocampus, a single-layer neuronal structure. Here we provide numerical evidence that the cytoarchitectonic complexity of other brain structures does not compromise the value of the ICA-separated LFP components, given that spatial sampling of LFP is representative. The spatial distribution of an LFP component may be quite complex due to folded and multilayered structure of the neuronal aggregate. Nevertheless, the time course of each LFP component is still a reliable postsynaptic convolution of spikes fired by a homogeneous afferent population. This claim is supported by preliminary experimental data obtained in the lateral geniculate nucleus of the awake monkey.

Theory of intermittency applied to classical pathological cases.

The classical theory of intermittency developed for return maps assumes uniform density of points reinjected from the chaotic to laminar region. Though it works fine in some model systems, there exist a number of so-called pathological cases characterized by a significant deviation of main characteristics from the values predicted on the basis of the uniform distribution. Recently, we reported on how the reinjection probability density (RPD) can be generalized. Here, we extend this methodology and apply it to different dynamical systems exhibiting anomalous type-II and type-III intermittencies. Estimation of the universal RPD is based on fitting a linear function to experimental data and requires no a priori knowledge on the dynamical model behind. We provide special fitting procedure that enables robust estimation of the RPD from relatively short data sets (dozens of points). Thus, the method is applicable for a wide variety of data sets including numerical simulations and real-life experiments. Estimated RPD enables analytic evaluation of the length of the laminar phase of intermittent behaviors. We show that the method copes well with dynamical systems exhibiting significantly different statistics reported in the literature. We also derive and classify characteristic relations between the mean laminar length and main controlling parameter in perfect agreement with data provided by numerical simulations.