Dynamic predictive coding: A model of hierarchical sequence learning and prediction in the neocortex.

We introduce dynamic predictive coding, a hierarchical model of spatiotemporal prediction and sequence learning in the neocortex. The model assumes that higher cortical levels modulate the temporal dynamics of lower levels, correcting their predictions of dynamics using prediction errors. As a result, lower levels form representations that encode sequences at shorter timescales (e.g., a single step) while higher levels form representations that encode sequences at longer timescales (e.g., an entire sequence). We tested this model using a two-level neural network, where the top-down modulation creates low-dimensional combinations of a set of learned temporal dynamics to explain input sequences. When trained on natural videos, the lower-level model neurons developed space-time receptive fields similar to those of simple cells in the primary visual cortex while the higher-level responses spanned longer timescales, mimicking temporal response hierarchies in the cortex. Additionally, the network's hierarchical sequence representation exhibited both predictive and postdictive effects resembling those observed in visual motion processing in humans (e.g., in the flash-lag illusion). When coupled with an associative memory emulating the role of the hippocampus, the model allowed episodic memories to be stored and retrieved, supporting cue-triggered recall of an input sequence similar to activity recall in the visual cortex. When extended to three hierarchical levels, the model learned progressively more abstract temporal representations along the hierarchy. Taken together, our results suggest that cortical processing and learning of sequences can be interpreted as dynamic predictive coding based on a hierarchical spatiotemporal generative model of the visual world.

Active Predictive Coding: A Unifying Neural Model for Active Perception, Compositional Learning, and Hierarchical Planning.

There is growing interest in predictive coding as a model of how the brain learns through predictions and prediction errors. Predictive coding models have traditionally focused on sensory coding and perception. Here we introduce active predictive coding (APC) as a unifying model for perception, action, and cognition. The APC model addresses important open problems in cognitive science and AI, including (1) how we learn compositional representations (e.g., part-whole hierarchies for equivariant vision) and (2) how we solve large-scale planning problems, which are hard for traditional reinforcement learning, by composing complex state dynamics and abstract actions from simpler dynamics and primitive actions. By using hypernetworks, self-supervised learning, and reinforcement learning, APC learns hierarchical world models by combining task-invariant state transition networks and task-dependent policy networks at multiple abstraction levels. We illustrate the applicability of the APC model to active visual perception and hierarchical planning. Our results represent, to our knowledge, the first proof-of-concept demonstration of a unified approach to addressing the part-whole learning problem in vision, the nested reference frames learning problem in cognition, and the integrated state-action hierarchy learning problem in reinforcement learning.

Spontaneous Decoding of the Timing and Content of Human Object Perception from Cortical Surface Recordings Reveals Complementary Information in the Event-Related Potential and Broadband Spectral Change.

The link between object perception and neural activity in visual cortical areas is a problem of fundamental importance in neuroscience. Here we show that electrical potentials from the ventral temporal cortical surface in humans contain sufficient information for spontaneous and near-instantaneous identification of a subject's perceptual state. Electrocorticographic (ECoG) arrays were placed on the subtemporal cortical surface of seven epilepsy patients. Grayscale images of faces and houses were displayed rapidly in random sequence. We developed a template projection approach to decode the continuous ECoG data stream spontaneously, predicting the occurrence, timing and type of visual stimulus. In this setting, we evaluated the independent and joint use of two well-studied features of brain signals, broadband changes in the frequency power spectrum of the potential and deflections in the raw potential trace (event-related potential; ERP). Our ability to predict both the timing of stimulus onset and the type of image was best when we used a combination of both the broadband response and ERP, suggesting that they capture different and complementary aspects of the subject's perceptual state. Specifically, we were able to predict the timing and type of 96% of all stimuli, with less than 5% false positive rate and a ~20ms error in timing.

Cortico-Cortical Interactions during Acquisition and Use of a Neuroprosthetic Skill.

A motor cortex-based brain-computer interface (BCI) creates a novel real world output directly from cortical activity. Use of a BCI has been demonstrated to be a learned skill that involves recruitment of neural populations that are directly linked to BCI control as well as those that are not. The nature of interactions between these populations, however, remains largely unknown. Here, we employed a data-driven approach to assess the interaction between both local and remote cortical areas during the use of an electrocorticographic BCI, a method which allows direct sampling of cortical surface potentials. Comparing the area controlling the BCI with remote areas, we evaluated relationships between the amplitude envelopes of band limited powers as well as non-linear phase-phase interactions. We found amplitude-amplitude interactions in the high gamma (HG, 70-150 Hz) range that were primarily located in the posterior portion of the frontal lobe, near the controlling site, and non-linear phase-phase interactions involving multiple frequencies (cross-frequency coupling between 8-11 Hz and 70-90 Hz) taking place over larger cortical distances. Further, strength of the amplitude-amplitude interactions decreased with time, whereas the phase-phase interactions did not. These findings suggest multiple modes of cortical communication taking place during BCI use that are specialized for function and depend on interaction distance.

Bayesian Inference and Online Learning in Poisson Neuronal Networks.

Motivated by the growing evidence for Bayesian computation in the brain, we show how a two-layer recurrent network of Poisson neurons can perform both approximate Bayesian inference and learning for any hidden Markov model. The lower-layer sensory neurons receive noisy measurements of hidden world states. The higher-layer neurons infer a posterior distribution over world states via Bayesian inference from inputs generated by sensory neurons. We demonstrate how such a neuronal network with synaptic plasticity can implement a form of Bayesian inference similar to Monte Carlo methods such as particle filtering. Each spike in a higher-layer neuron represents a sample of a particular hidden world state. The spiking activity across the neural population approximates the posterior distribution over hidden states. In this model, variability in spiking is regarded not as a nuisance but as an integral feature that provides the variability necessary for sampling during inference. We demonstrate how the network can learn the likelihood model, as well as the transition probabilities underlying the dynamics, using a Hebbian learning rule. We present results illustrating the ability of the network to perform inference and learning for arbitrary hidden Markov models.

Distributed cortical adaptation during learning of a brain-computer interface task.

The majority of subjects who attempt to learn control of a brain-computer interface (BCI) can do so with adequate training. Much like when one learns to type or ride a bicycle, BCI users report transitioning from a deliberate, cognitively focused mindset to near automatic control as training progresses. What are the neural correlates of this process of BCI skill acquisition? Seven subjects were implanted with electrocorticography (ECoG) electrodes and had multiple opportunities to practice a 1D BCI task. As subjects became proficient, strong initial task-related activation was followed by lessening of activation in prefrontal cortex, premotor cortex, and posterior parietal cortex, areas that have previously been implicated in the cognitive phase of motor sequence learning and abstract task learning. These results demonstrate that, although the use of a BCI only requires modulation of a local population of neurons, a distributed network of cortical areas is involved in the acquisition of BCI proficiency.

The physiology of perception in human temporal lobe is specialized for contextual novelty.

The human ventral temporal cortex has regions that are known to selectively process certain categories of visual inputs; they are specialized for the content ("faces," "places," "tools") and not the form ("line," "patch") of the image being seen. In our study, human patients with implanted electrocorticography (ECoG) electrode arrays were shown sequences of simple face and house pictures. We quantified neuronal population activity, finding robust face-selective sites on the fusiform gyrus and house-selective sites on the lingual/parahippocampal gyri. The magnitude and timing of single trials were compared between novel ("house-face") and repeated ("face-face") stimulus-type responses. More than half of the category-selective sites showed significantly greater total activity for novel stimulus class. Approximately half of the face-selective sites (and none of the house-selective sites) showed significantly faster latency to peak (∼ 50 ms) for novel stimulus class. This establishes subregions within category-selective areas that are differentially tuned to novelty in sequential context, where novel stimuli are processed faster in some regions, and with increased activity in others.

Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations.

We illustrate a general principal of electrical potential measurements from the surface of the cerebral cortex, by revisiting and reanalyzing experimental work from the visual, language and motor systems. A naive decomposition technique of electrocorticographic power spectral measurements reveals that broadband spectral changes reliably track task engagement. These broadband changes are shown to be a generic correlate of local cortical function across a variety of brain areas and behavioral tasks. Furthermore, they fit a power-law form that is consistent with simple models of the dendritic integration of asynchronous local population firing. Because broadband spectral changes covary with diverse perceptual and behavioral states on the timescale of 20-50 ms, they provide a powerful and widely applicable experimental tool.

A sensory-motor theory of the neocortex.

Recent neurophysiological and neuroanatomical studies suggest a close interaction between sensory and motor processes across the neocortex. Here, I propose that the neocortex implements active predictive coding (APC): each cortical area estimates both latent sensory states and actions (including potentially abstract actions internal to the cortex), and the cortex as a whole predicts the consequences of actions at multiple hierarchical levels. Feedback from higher areas modulates the dynamics of state and action networks in lower areas. I show how the same APC architecture can explain (1) how we recognize an object and its parts using eye movements, (2) why perception seems stable despite eye movements, (3) how we learn compositional representations, for example, part-whole hierarchies, (4) how complex actions can be planned using simpler actions, and (5) how we form episodic memories of sensory-motor experiences and learn abstract concepts such as a family tree. I postulate a mapping of the APC model to the laminar architecture of the cortex and suggest possible roles for cortico-cortical and cortico-subcortical pathways.

Neurocomputational mechanisms involved in adaptation to fluctuating intentions of others.

Humans frequently interact with agents whose intentions can fluctuate between competition and cooperation over time. It is unclear how the brain adapts to fluctuating intentions of others when the nature of the interactions (to cooperate or compete) is not explicitly and truthfully signaled. Here, we use model-based fMRI and a task in which participants thought they were playing with another player. In fact, they played with an algorithm that alternated without signaling between cooperative and competitive strategies. We show that a neurocomputational mechanism with arbitration between competitive and cooperative experts outperforms other learning models in predicting choice behavior. At the brain level, the fMRI results show that the ventral striatum and ventromedial prefrontal cortex track the difference of reliability between these experts. When attributing competitive intentions, we find increased coupling between these regions and a network that distinguishes prediction errors related to competition and cooperation. These findings provide a neurocomputational account of how the brain arbitrates dynamically between cooperative and competitive intentions when making adaptive social decisions.

Cortical activity during motor execution, motor imagery, and imagery-based online feedback.

Imagery of motor movement plays an important role in learning of complex motor skills, from learning to serve in tennis to perfecting a pirouette in ballet. What and where are the neural substrates that underlie motor imagery-based learning? We measured electrocorticographic cortical surface potentials in eight human subjects during overt action and kinesthetic imagery of the same movement, focusing on power in "high frequency" (76-100 Hz) and "low frequency" (8-32 Hz) ranges. We quantitatively establish that the spatial distribution of local neuronal population activity during motor imagery mimics the spatial distribution of activity during actual motor movement. By comparing responses to electrocortical stimulation with imagery-induced cortical surface activity, we demonstrate the role of primary motor areas in movement imagery. The magnitude of imagery-induced cortical activity change was approximately 25% of that associated with actual movement. However, when subjects learned to use this imagery to control a computer cursor in a simple feedback task, the imagery-induced activity change was significantly augmented, even exceeding that of overt movement.

Recursive neural programs: A differentiable framework for learning compositional part-whole hierarchies and image grammars.

Human vision, thought, and planning involve parsing and representing objects and scenes using structured representations based on part-whole hierarchies. Computer vision and machine learning researchers have recently sought to emulate this capability using neural networks, but a generative model formulation has been lacking. Generative models that leverage compositionality, recursion, and part-whole hierarchies are thought to underlie human concept learning and the ability to construct and represent flexible mental concepts. We introduce Recursive Neural Programs (RNPs), a neural generative model that addresses the part-whole hierarchy learning problem by modeling images as hierarchical trees of probabilistic sensory-motor programs. These programs recursively reuse learned sensory-motor primitives to model an image within different spatial reference frames, enabling hierarchical composition of objects from parts and implementing a grammar for images. We show that RNPs can learn part-whole hierarchies for a variety of image datasets, allowing rich compositionality and intuitive parts-based explanations of objects. Our model also suggests a cognitive framework for understanding how human brains can potentially learn and represent concepts in terms of recursively defined primitives and their relations with each other.

Expressive probabilistic sampling in recurrent neural networks.

In sampling-based Bayesian models of brain function, neural activities are assumed to be samples from probability distributions that the brain uses for probabilistic computation. However, a comprehensive understanding of how mechanistic models of neural dynamics can sample from arbitrary distributions is still lacking. We use tools from functional analysis and stochastic differential equations to explore the minimum architectural requirements for $\textit{recurrent}$ neural circuits to sample from complex distributions. We first consider the traditional sampling model consisting of a network of neurons whose outputs directly represent the samples (sampler-only network). We argue that synaptic current and firing-rate dynamics in the traditional model have limited capacity to sample from a complex probability distribution. We show that the firing rate dynamics of a recurrent neural circuit with a separate set of output units can sample from an arbitrary probability distribution. We call such circuits reservoir-sampler networks (RSNs). We propose an efficient training procedure based on denoising score matching that finds recurrent and output weights such that the RSN implements Langevin sampling. We empirically demonstrate our model's ability to sample from several complex data distributions using the proposed neural dynamics and discuss its applicability to developing the next generation of sampling-based brain models.

Responses of Cortical Neurons to Intracortical Microstimulation in Awake Primates.

Intracortical microstimulation (ICMS) is commonly used in many experimental and clinical paradigms; however, its effects on the activation of neurons are still not completely understood. To document the responses of cortical neurons in awake nonhuman primates to stimulation, we recorded single-unit activity while delivering single-pulse stimulation via Utah arrays implanted in primary motor cortex (M1) of three macaque monkeys. Stimuli between 5 and 50 µA delivered to single channels reliably evoked spikes in neurons recorded throughout the array with delays of up to 12 ms. ICMS pulses also induced a period of inhibition lasting up to 150 ms that typically followed the initial excitatory response. Higher current amplitudes led to a greater probability of evoking a spike and extended the duration of inhibition. The likelihood of evoking a spike in a neuron was dependent on the spontaneous firing rate as well as the delay between its most recent spike time and stimulus onset. Tonic repetitive stimulation between 2 and 20 Hz often modulated both the probability of evoking spikes and the duration of inhibition; high-frequency stimulation was more likely to change both responses. On a trial-by-trial basis, whether a stimulus evoked a spike did not affect the subsequent inhibitory response; however, their changes over time were often positively or negatively correlated. Our results document the complex dynamics of cortical neural responses to electrical stimulation that need to be considered when using ICMS for scientific and clinical applications.

Local field potentials and single unit dynamics in motor cortex of unconstrained macaques during different behavioral states.

Different sleep stages have been shown to be vital for a variety of brain functions, including learning, memory, and skill consolidation. However, our understanding of neural dynamics during sleep and the role of prominent LFP frequency bands remain incomplete. To elucidate such dynamics and differences between behavioral states we collected multichannel LFP and spike data in primary motor cortex of unconstrained macaques for up to 24 h using a head-fixed brain-computer interface (Neurochip3). Each 8-s bin of time was classified into awake-moving (Move), awake-resting (Rest), REM sleep (REM), or non-REM sleep (NREM) by using dimensionality reduction and clustering on the average spectral density and the acceleration of the head. LFP power showed high delta during NREM, high theta during REM, and high beta when the animal was awake. Cross-frequency phase-amplitude coupling typically showed higher coupling during NREM between all pairs of frequency bands. Two notable exceptions were high delta-high gamma and theta-high gamma coupling during Move, and high theta-beta coupling during REM. Single units showed decreased firing rate during NREM, though with increased short ISIs compared to other states. Spike-LFP synchrony showed high delta synchrony during Move, and higher coupling with all other frequency bands during NREM. These results altogether reveal potential roles and functions of different LFP bands that have previously been unexplored.

Emergent behaviour and neural dynamics in artificial agents tracking odour plumes.

Tracking an odour plume to locate its source under variable wind and plume statistics is a complex task. Flying insects routinely accomplish such tracking, often over long distances, in pursuit of food or mates. Several aspects of this remarkable behaviour and its underlying neural circuitry have been studied experimentally. Here we take a complementary in silico approach to develop an integrated understanding of their behaviour and neural computations. Specifically, we train artificial recurrent neural network agents using deep reinforcement learning to locate the source of simulated odour plumes that mimic features of plumes in a turbulent flow. Interestingly, the agents' emergent behaviours resemble those of flying insects, and the recurrent neural networks learn to compute task-relevant variables with distinct dynamic structures in population activity. Our analyses put forward a testable behavioural hypothesis for tracking plumes in changing wind direction, and we provide key intuitions for memory requirements and neural dynamics in odour plume tracking.

A Markov model of the Indus script.

Although no historical information exists about the Indus civilization (flourished ca. 2600-1900 B.C.), archaeologists have uncovered about 3,800 short samples of a script that was used throughout the civilization. The script remains undeciphered, despite a large number of attempts and claimed decipherments over the past 80 years. Here, we propose the use of probabilistic models to analyze the structure of the Indus script. The goal is to reveal, through probabilistic analysis, syntactic patterns that could point the way to eventual decipherment. We illustrate the approach using a simple Markov chain model to capture sequential dependencies between signs in the Indus script. The trained model allows new sample texts to be generated, revealing recurring patterns of signs that could potentially form functional subunits of a possible underlying language. The model also provides a quantitative way of testing whether a particular string belongs to the putative language as captured by the Markov model. Application of this test to Indus seals found in Mesopotamia and other sites in West Asia reveals that the script may have been used to express different content in these regions. Finally, we show how missing, ambiguous, or unreadable signs on damaged objects can be filled in with most likely predictions from the model. Taken together, our results indicate that the Indus script exhibits rich synactic structure and the ability to represent diverse content. both of which are suggestive of a linguistic writing system rather than a nonlinguistic symbol system.

Learning neural decoders without labels using multiple data streams.

Objective.Recent advances in neural decoding have accelerated the development of brain-computer interfaces aimed at assisting users with everyday tasks such as speaking, walking, and manipulating objects. However, current approaches for training neural decoders commonly require large quantities of labeled data, which can be laborious or infeasible to obtain in real-world settings. Alternatively, self-supervised models that share self-generated pseudo-labels between two data streams have shown exceptional performance on unlabeled audio and video data, but it remains unclear how well they extend to neural decoding.Approach.We learn neural decoders without labels by leveraging multiple simultaneously recorded data streams, including neural, kinematic, and physiological signals. Specifically, we apply cross-modal, self-supervised deep clustering to train decoders that can classify movements from brain recordings. After training, we then isolate the decoders for each input data stream and compare the accuracy of decoders trained using cross-modal deep clustering against supervised and unimodal, self-supervised models.Main results.We find that sharing pseudo-labels between two data streams during training substantially increases decoding performance compared to unimodal, self-supervised models, with accuracies approaching those of supervised decoders trained on labeled data. Next, we extend cross-modal decoder training to three or more modalities, achieving state-of-the-art neural decoding accuracy that matches or slightly exceeds the performance of supervised models.Significance.We demonstrate that cross-modal, self-supervised decoding can be applied to train neural decoders when few or no labels are available and extend the cross-modal framework to share information among three or more data streams, further improving self-supervised training.

An Immersive Virtual Reality Platform Integrating Human ECOG & sEEG: Implementation & Noise Analysis.

Virtual reality (VR) offers a robust platform for human behavioral neuroscience, granting unprecedented experimental control over every aspect of an immersive and interactive visual environment. VR experiments have already integrated non-invasive neural recording modalities such as EEG and functional MRI to explore the neural correlates of human behavior and cognition. Integration with implanted electrodes would enable significant increase in spatial and temporal resolution of recorded neural signals and the option of direct brain stimulation for neurofeedback. In this paper, we discuss the first such implementation of a VR platform with implanted electrocorticography (ECoG) and stereo-electroencephalography ( sEEG) electrodes in human, in-patient subjects. Noise analyses were performed to evaluate the effect of the VR headset on neural data collected in two VR-naive subjects, one child and one adult, including both ECOG and sEEG electrodes. Results demonstrate an increase in line noise power (57-63Hz) while wearing the VR headset that is mitigated effectively by common average referencing (CAR), and no significant change in the noise floor bandpower (125-240Hz). To our knowledge, this study represents first demonstrations of VR immersion during invasive neural recording with in-patient human subjects. Clinical Relevance- Immersive virtual reality tasks were well-tolerated and the quality of clinical neural signals preserved during VR immersion with two in-patient invasive neural recording subjects.

AJILE12: Long-term naturalistic human intracranial neural recordings and pose.

Understanding the neural basis of human movement in naturalistic scenarios is critical for expanding neuroscience research beyond constrained laboratory paradigms. Here, we describe our Annotated Joints in Long-term Electrocorticography for 12 human participants (AJILE12) dataset, the largest human neurobehavioral dataset that is publicly available; the dataset was recorded opportunistically during passive clinical epilepsy monitoring. AJILE12 includes synchronized intracranial neural recordings and upper body pose trajectories across 55 semi-continuous days of naturalistic movements, along with relevant metadata, including thousands of wrist movement events and annotated behavioral states. Neural recordings are available at 500 Hz from at least 64 electrodes per participant, for a total of 1280 hours. Pose trajectories at 9 upper-body keypoints were estimated from 118 million video frames. To facilitate data exploration and reuse, we have shared AJILE12 on The DANDI Archive in the Neurodata Without Borders (NWB) data standard and developed a browser-based dashboard.