Efficient models of cortical activity via local dynamic equilibria and coarse-grained interactions.

Biologically detailed models of brain circuitry are challenging to build and simulate due to the large number of neurons, their complex interactions, and the many unknown physiological parameters. Simplified mathematical models are more tractable, but harder to evaluate when too far removed from neuroanatomy/physiology. We propose that a multiscale model, coarse-grained (CG) while preserving local biological details, offers the best balance between biological realism and computability. This paper presents such a model. Generally, CG models focus on the interaction between groups of neurons-here termed "pixels"-rather than individual cells. In our case, dynamics are alternately updated at intra- and interpixel scales, with one informing the other, until convergence to equilibrium is achieved on both scales. An innovation is how we exploit the underlying biology: Taking advantage of the similarity in local anatomical structures across large regions of the cortex, we model intrapixel dynamics as a single dynamical system driven by "external" inputs. These inputs vary with events external to the pixel, but their ranges can be estimated a priori. Precomputing and tabulating all potential local responses speed up the updating procedure significantly compared to direct multiscale simulation. We illustrate our methodology using a model of the primate visual cortex. Except for local neuron-to-neuron variability (necessarily lost in any CG approximation) our model reproduces various features of large-scale network models at a tiny fraction of the computational cost. These include neuronal responses as a consequence of their orientation selectivity, a primary function of visual neurons.

Growth and depletion in linear stochastic reaction networks.

This paper is about a class of stochastic reaction networks. Of interest are the dynamics of interconversion among a finite number of substances through reactions that consume some of the substances and produce others. The models we consider are continuous-time Markov jump processes, intended as idealizations of a broad class of biological networks. Reaction rates depend linearly on "enzymes," which are among the substances produced, and a reaction can occur only in the presence of sufficient upstream material. We present rigorous results for this class of stochastic dynamical systems, the mean-field behaviors of which are described by ordinary differential equations (ODEs). Under the assumption of exponential network growth, we identify certain ODE solutions as being potentially traceable and give conditions on network trajectories which, when rescaled, can with high probability be approximated by these ODE solutions. This leads to a complete characterization of the ω-limit sets of such network solutions (as points or random tori). Dimension reduction is noted depending on the number of enzymes. The second half of this paper is focused on depletion dynamics, i.e., dynamics subsequent to the "phase transition" that occurs when one of the substances becomes unavailable. The picture can be complex, for the depleted substance can be produced intermittently through other network reactions. Treating the model as a slow-fast system, we offer a mean-field description, a first step to understanding what we believe is one of the most natural bifurcations for reaction networks.

A theory of direction selectivity for macaque primary visual cortex.

This paper offers a theory for the origin of direction selectivity (DS) in the macaque primary visual cortex, V1. DS is essential for the perception of motion and control of pursuit eye movements. In the macaque visual pathway, neurons with DS first appear in V1, in the Simple cell population of the Magnocellular input layer 4Cα. The lateral geniculate nucleus (LGN) cells that project to these cortical neurons, however, are not direction selective. We hypothesize that DS is initiated in feed-forward LGN input, in the summed responses of LGN cells afferent to a cortical cell, and it is achieved through the interplay of 1) different visual response dynamics of ON and OFF LGN cells and 2) the wiring of ON and OFF LGN neurons to cortex. We identify specific temporal differences in the ON/OFF pathways that, together with item 2, produce distinct response time courses in separated subregions; analysis and simulations confirm the efficacy of the mechanisms proposed. To constrain the theory, we present data on Simple cells in layer 4Cα in response to drifting gratings. About half of the cells were found to have high DS, and the DS was broadband in spatial and temporal frequency (SF and TF). The proposed theory includes a complete analysis of how stimulus features such as SF and TF interact with ON/OFF dynamics and LGN-to-cortex wiring to determine the preferred direction and magnitude of DS.

A Computational Model of Direction Selectivity in Macaque V1 Cortex Based on Dynamic Differences between On and Off Pathways.

This paper is about neural mechanisms of direction selectivity (DS) in macaque primary visual cortex, V1. We present data (on male macaque) showing strong DS in a majority of simple cells in V1 layer 4Cα, the cortical layer that receives direct afferent input from the magnocellular division of the lateral geniculate nucleus (LGN). Magnocellular LGN cells are not direction-selective. To understand the mechanisms of DS, we built a large-scale, recurrent model of spiking neurons called DSV1. Like its predecessors, DSV1 reproduces many visual response properties of V1 cells including orientation selectivity. Two important new features of DSV1 are (1) DS is initiated by small, consistent dynamic differences in the visual responses of OFF and ON Magnocellular LGN cells, and (2) DS in the responses of most model simple cells is increased over those of their feedforward inputs; this increase is achieved through dynamic interaction of feedforward and intracortical synaptic currents without the use of intracortical direction-specific connections. The DSV1 model emulates experimental data in the following ways: (1) most 4Cα Simple cells were highly direction-selective but 4Cα Complex cells were not; (2) the preferred directions of the model's direction-selective Simple cells were invariant with spatial and temporal frequency (TF); (3) the distribution of the preferred/opposite ratio across the model's population of cells was very close to that found in experiments. The strong quantitative agreement between DS in data and in model simulations suggests that the neural mechanisms of DS in DSV1 may be similar to those in the real visual cortex.SIGNIFICANCE STATEMENT Motion perception is a vital part of our visual experience of the world. In monkeys, whose vision resembles that of humans, the neural computation of the direction of a moving target starts in the primary visual cortex, V1, in layer 4Cα that receives input from the eye through the lateral geniculate nucleus (LGN). How direction selectivity (DS) is generated in layer 4Cα is an outstanding unsolved problem in theoretical neuroscience. In this paper, we offer a solution based on plausible biological mechanisms. We present a new large-scale circuit model in which DS originates from slightly different LGN ON/OFF response time-courses and is enhanced in cortex without the need for direction-specific intracortical connections. The model's DS is in quantitative agreement with experiments.

Origin of exponential growth in nonlinear reaction networks.

Exponentially growing systems are prevalent in nature, spanning all scales from biochemical reaction networks in single cells to food webs of ecosystems. How exponential growth emerges in nonlinear systems is mathematically unclear. Here, we describe a general theoretical framework that reveals underlying principles of long-term growth: scalability of flux functions and ergodicity of the rescaled systems. Our theory shows that nonlinear fluxes can generate not only balanced growth but also oscillatory or chaotic growth modalities, explaining nonequilibrium dynamics observed in cell cycles and ecosystems. Our mathematical framework is broadly useful in predicting long-term growth rates from natural and synthetic networks, analyzing the effects of system noise and perturbations, validating empirical and phenomenological laws on growth rate, and studying autocatalysis and network evolution.

Unraveling the mechanisms of surround suppression in early visual processing.

This paper uses mathematical modeling to study the mechanisms of surround suppression in the primate visual cortex. We present a large-scale neural circuit model consisting of three interconnected components: LGN and two input layers (Layer 4Ca and Layer 6) of the primary visual cortex V1, covering several hundred hypercolumns. Anatomical structures are incorporated and physiological parameters from realistic modeling work are used. The remaining parameters are chosen to produce model outputs that emulate experimentally observed size-tuning curves. Our two main results are: (i) we discovered the character of the long-range connections in Layer 6 responsible for surround effects in the input layers; and (ii) we showed that a net-inhibitory feedback, i.e., feedback that excites I-cells more than E-cells, from Layer 6 to Layer 4 is conducive to producing surround properties consistent with experimental data. These results are obtained through parameter selection and model analysis. The effects of nonlinear recurrent excitation and inhibition are also discussed. A feature that distinguishes our model from previous modeling work on surround suppression is that we have tried to reproduce realistic lengthscales that are crucial for quantitative comparison with data. Due to its size and the large number of unknown parameters, the model is computationally challenging. We demonstrate a strategy that involves first locating baseline values for relevant parameters using a linear model, followed by the introduction of nonlinearities where needed. We find such a methodology effective, and propose it as a possibility in the modeling of complex biological systems.

A data-informed mean-field approach to mapping of cortical parameter landscapes.

Constraining the many biological parameters that govern cortical dynamics is computationally and conceptually difficult because of the curse of dimensionality. This paper addresses these challenges by proposing (1) a novel data-informed mean-field (MF) approach to efficiently map the parameter space of network models; and (2) an organizing principle for studying parameter space that enables the extraction biologically meaningful relations from this high-dimensional data. We illustrate these ideas using a large-scale network model of the Macaque primary visual cortex. Of the 10-20 model parameters, we identify 7 that are especially poorly constrained, and use the MF algorithm in (1) to discover the firing rate contours in this 7D parameter cube. Defining a "biologically plausible" region to consist of parameters that exhibit spontaneous Excitatory and Inhibitory firing rates compatible with experimental values, we find that this region is a slightly thickened codimension-1 submanifold. An implication of this finding is that while plausible regimes depend sensitively on parameters, they are also robust and flexible provided one compensates appropriately when parameters are varied. Our organizing principle for conceptualizing parameter dependence is to focus on certain 2D parameter planes that govern lateral inhibition: Intersecting these planes with the biologically plausible region leads to very simple geometric structures which, when suitably scaled, have a universal character independent of where the intersections are taken. In addition to elucidating the geometry of the plausible region, this invariance suggests useful approximate scaling relations. Our study offers, for the first time, a complete characterization of the set of all biologically plausible parameters for a detailed cortical model, which has been out of reach due to the high dimensionality of parameter space.

Chaotic heteroclinic networks as models of switching behavior in biological systems.

Key features of biological activity can often be captured by transitions between a finite number of semi-stable states that correspond to behaviors or decisions. We present here a broad class of dynamical systems that are ideal for modeling such activity. The models we propose are chaotic heteroclinic networks with nontrivial intersections of stable and unstable manifolds. Due to the sensitive dependence on initial conditions, transitions between states are seemingly random. Dwell times, exit distributions, and other transition statistics can be built into the model through geometric design and can be controlled by tunable parameters. To test our model's ability to simulate realistic biological phenomena, we turned to one of the most studied organisms, C. elegans, well known for its limited behavioral states. We reconstructed experimental data from two laboratories, demonstrating the model's ability to quantitatively reproduce dwell times and transition statistics under a variety of conditions. Stochastic switching between dominant states in complex dynamical systems has been extensively studied and is often modeled as Markov chains. As an alternative, we propose here a new paradigm, namely, chaotic heteroclinic networks generated by deterministic rules (without the necessity for noise). Chaotic heteroclinic networks can be used to model systems with arbitrary architecture and size without a commensurate increase in phase dimension. They are highly flexible and able to capture a wide range of transition characteristics that can be adjusted through control parameters.

Malleability of gamma rhythms enhances population-level correlations.

An important problem in systems neuroscience is to understand how information is communicated among brain regions, and it has been proposed that communication is mediated by neuronal oscillations, such as rhythms in the gamma band. We sought to investigate this idea by using a network model with two components, a source (sending) and a target (receiving) component, both built to resemble local populations in the cerebral cortex. To measure the effectiveness of communication, we used population-level correlations in spike times between the source and target. We found that after correcting for a response time that is independent of initial conditions, spike-time correlations between the source and target are significant, due in large measure to the alignment of firing events in their gamma rhythms. But, we also found that regular oscillations cannot produce the results observed in our model simulations of cortical neurons. Surprisingly, it is the irregularity of gamma rhythms, the absence of internal clocks, together with the malleability of these rhythms and their tendency to align with external pulses - features that are known to be present in gamma rhythms in the real cortex - that produced the results observed. These findings and the mechanistic explanations we offered are our primary results. Our secondary result is a mathematical relationship between correlations and the sizes of the samples used for their calculation. As improving technology enables recording simultaneously from increasing numbers of neurons, this relationship could be useful for interpreting results from experimental recordings.

A case study in the functional consequences of scaling the sizes of realistic cortical models.

Neuroscience models come in a wide range of scales and specificity, from mean-field rate models to large-scale networks of spiking neurons. There are potential trade-offs between simplicity and realism, versatility and computational speed. This paper is about large-scale cortical network models, and the question we address is one of scalability: would scaling down cell density impact a network's ability to reproduce cortical dynamics and function? We investigated this problem using a previously constructed realistic model of the monkey visual cortex that is true to size. Reducing cell density gradually up to 50-fold, we studied changes in model behavior. Size reduction without parameter adjustment was catastrophic. Surprisingly, relatively minor compensation in synaptic weights guided by a theoretical algorithm restored mean firing rates and basic function such as orientation selectivity to models 10-20 times smaller than the real cortex. Not all was normal in the reduced model cortices: intracellular dynamics acquired a character different from that of real neurons, and while the ability to relay feedforward inputs remained intact, reduced models showed signs of deficiency in functions that required dynamical interaction among cortical neurons. These findings are not confined to models of the visual cortex, and modelers should be aware of potential issues that accompany size reduction. Broader implications of this study include the importance of homeostatic maintenance of firing rates, and the functional consequences of feedforward versus recurrent dynamics, ideas that may shed light on other species and on systems suffering cell loss.

Contrast response in a comprehensive network model of macaque V1.

The response to contrast is one of the most important functions of the macaque primary visual cortex, V1, but up to now there has not been an adequate theory for it. To fill this gap in our understanding of cortical function, we built and analyzed a new large-scale, biologically constrained model of the input layer, 4Cα, of macaque V1. We called the new model CSY2. We challenged CSY2 with a three-parameter family of visual stimuli that varied in contrast, orientation, and spatial frequency. CSY2 accurately simulated experimental data and made many new predictions. It accounted for 1) the shapes of firing-rate-versus-contrast functions, 2) orientation and spatial frequency tuning versus contrast, and 3) the approximate contrast-invariance of cortical activity maps. Post-analysis revealed that the mechanisms that were needed to produce the successful simulations of contrast response included strong recurrent excitation and inhibition that find dynamic equilibria across the cortical surface, dynamic feedback between L6 and L4, and synaptic dynamics like inhibitory synaptic depression.

Rhythm and Synchrony in a Cortical Network Model.

We studied mechanisms for cortical gamma-band activity in the cerebral cortex and identified neurobiological factors that affect such activity. This was done by analyzing the behavior of a previously developed, data-driven, large-scale network model that simulated many visual functions of monkey V1 cortex (Chariker et al., 2016). Gamma activity was an emergent property of the model. The model's gamma activity, like that of the real cortex, was (1) episodic, (2) variable in frequency and phase, and (3) graded in power with stimulus variables like orientation. The spike firing of the model's neuronal population was only partially synchronous during multiple firing events (MFEs) that occurred at gamma rates. Detailed analysis of the model's MFEs showed that gamma-band activity was multidimensional in its sources. Most spikes were evoked by excitatory inputs. A large fraction of these inputs came from recurrent excitation within the local circuit, but feedforward and feedback excitation also contributed, either through direct pulsing or by raising the overall baseline. Inhibition was responsible for ending MFEs, but disinhibition led directly to only a small minority of the synchronized spikes. As a potential explanation for the wide range of gamma characteristics observed in different parts of cortex, we found that the relative rise times of AMPA and GABA synaptic conductances have a strong effect on the degree of synchrony in gamma.SIGNIFICANCE STATEMENT Canonical computations used throughout the cerebral cortex are performed in primary visual cortex (V1). Providing theoretical mechanisms for these computations will advance understanding of computation throughout cortex. We studied one dynamical feature, gamma-band rhythms, in a large-scale, data-driven, computational model of monkey V1. Our most significant conclusion is that the sources of gamma band activity are multidimensional. A second major finding is that the relative rise times of excitatory and inhibitory synaptic potentials have strong effects on spike synchrony and peak gamma band power. Insight gained from studying our V1 model can shed light on the functions of other cortical regions.

How well do reduced models capture the dynamics in models of interacting neurons?

This paper introduces a class of stochastic models of interacting neurons with emergent dynamics similar to those seen in local cortical populations. Rigorous results on existence and uniqueness of nonequilibrium steady states are proved. These network models are then compared to very simple reduced models driven by the same mean excitatory and inhibitory currents. Discrepancies in firing rates between network and reduced models are investigated and explained by correlations in spiking, or partial synchronization, working in concert with "nonlinearities" in the time evolution of membrane potentials. The use of simple random walks and their first passage times to simulate fluctuations in neuronal membrane potentials and interspike times is also considered.

An SIQ delay differential equations model for disease control via isolation.

Infectious diseases are among the most prominent threats to mankind. When preventive health care cannot be provided, a viable means of disease control is the isolation of individuals who may be infected. To study the impact of isolation, we propose a system of delay differential equations and offer our model analysis based on the geometric theory of semi-flows. Calibrating the response to an outbreak in terms of the fraction of infectious individuals isolated and the speed with which this is done, we deduce the minimum response required to curb an incipient outbreak, and predict the ensuing endemic state should the infection continue to spread.

Orientation Selectivity from Very Sparse LGN Inputs in a Comprehensive Model of Macaque V1 Cortex.

A new computational model of the primary visual cortex (V1) of the macaque monkey was constructed to reconcile the visual functions of V1 with anatomical data on its LGN input, the extreme sparseness of which presented serious challenges to theoretically sound explanations of cortical function. We demonstrate that, even with such sparse input, it is possible to produce robust orientation selectivity, as well as continuity in the orientation map. We went beyond that to find plausible dynamic regimes of our new model that emulate simultaneously experimental data for a wide range of V1 phenomena, beginning with orientation selectivity but also including diversity in neuronal responses, bimodal distributions of the modulation ratio (the simple/complex classification), and dynamic signatures, such as gamma-band oscillations. Intracortical interactions play a major role in all aspects of the visual functions of the model. SIGNIFICANCE STATEMENT: We present the first realistic model that has captured the sparseness of magnocellular LGN inputs to the macaque primary visual cortex and successfully derived orientation selectivity from them. Three implications are (1) even in input layers to the visual cortex, the system is less feedforward and more dominated by intracortical signals than previously thought, (2) interactions among cortical neurons in local populations produce dynamics not explained by single neurons, and (3) such dynamics are important for function. Our model also shows that a comprehensive picture is necessary to explain function, because different visual properties are related. This study points to the need for paradigm shifts in neuroscience modeling: greater emphasis on population dynamics and, where possible, a move toward data-driven, comprehensive models.

Dynamic Signal Tracking in a Simple V1 Spiking Model.

This work is part of an effort to understand the neural basis for our visual system's ability, or failure, to accurately track moving visual signals. We consider here a ring model of spiking neurons, intended as a simplified computational model of a single hypercolumn of the primary visual cortex of primates. Signals that consist of edges with time-varying orientations localized in space are considered. Our model is calibrated to produce spontaneous and driven firing rates roughly consistent with experiments, and our two main findings, for which we offer dynamical explanation on the level of neuronal interactions, are the following. First, we have documented consistent transient overshoots in signal perception following signal switches due to emergent interactions of the E- and I-populations. Second, for continuously moving signals, we have found that accuracy is considerably lower at reversals of orientation than when continuing in the same direction (as when the signal is a rotating bar). To measure performance, we use two metrics, called fidelity and reliability, to compare signals reconstructed by the system to the ones presented and assess trial-to-trial variability. We propose that the same population mechanisms responsible for orientation selectivity also impose constraints on dynamic signal tracking that manifest in perception failures consistent with psychophysical observations.

Emergent spike patterns in neuronal populations.

This numerical study documents and analyzes emergent spiking behavior in local neuronal populations. Emphasis is given to a phenomenon we call clustering, by which we refer to a tendency of random groups of neurons large and small to spontaneously coordinate their spiking activity in some fashion. Using a sparsely connected network of integrate-and-fire neurons, we demonstrate that spike clustering occurs ubiquitously in both high firing and low firing regimes. As a practical tool for quantifying such spike patterns, we propose a simple scheme with two parameters, one setting the temporal scale and the other the amount of deviation from the mean to be regarded as significant. Viewing population activity as a sequence of events, meaning relatively brief durations of elevated spiking, separated by inter-event times, we observe that background activity tends to give rise to extremely broad distributions of event sizes and inter-event times, while driving a system imposes a certain regularity on its inter-event times, producing a rhythm consistent with broad-band gamma oscillations. We note also that event sizes and inter-event times decorrelate very quickly. Dynamical analyses supported by numerical evidence are offered.

Modeling temporal networks using random itineraries.

We propose a procedure to generate dynamical networks with bursty, possibly repetitive and correlated temporal behaviors. Regarding any weighted directed graph as being composed of the accumulation of paths between its nodes, our construction uses random walks of variable length to produce time-extended structures with adjustable features. The procedure is first described in a general framework. It is then illustrated in a case study inspired by a transportation system for which the resulting synthetic network is shown to accurately mimic the empirical phenomenology.

Emergent dynamics in a model of visual cortex.

This paper proposes that the network dynamics of the mammalian visual cortex are highly structured and strongly shaped by temporally localized barrages of excitatory and inhibitory firing we call 'multiple-firing events' (MFEs). Our proposal is based on careful study of a network of spiking neurons built to reflect the coarse physiology of a small patch of layer 2/3 of V1. When appropriately benchmarked this network is capable of reproducing the qualitative features of a range of phenomena observed in the real visual cortex, including spontaneous background patterns, orientation-specific responses, surround suppression and gamma-band oscillations. Detailed investigation into the relevant regimes reveals causal relationships among dynamical events driven by a strong competition between the excitatory and inhibitory populations. It suggests that along with firing rates, MFE characteristics can be a powerful signature of a regime. Testable predictions based on model observations and dynamical analysis are proposed.

Dynamics of spiking neurons: between homogeneity and synchrony.

Randomly connected networks of neurons driven by Poisson inputs are often assumed to produce "homogeneous" dynamics, characterized by largely independent firing and approximable by diffusion processes. At the same time, it is well known that such networks can fire synchronously. Between these two much studied scenarios lies a vastly complex dynamical landscape that is relatively unexplored. In this paper, we discuss a phenomenon which commonly manifests in these intermediate regimes, namely brief spurts of spiking activity which we call multiple firing events (MFE). These events do not depend on structured network architecture nor on structured input; they are an emergent property of the system. We came upon them in an earlier modeling paper, in which we discovered, through a careful benchmarking process, that MFEs are the single most important dynamical mechanism behind many of the V1 phenomena we were able to replicate. In this paper we explain in a simpler setting how MFEs come about, as well as their potential dynamic consequences. Although the mechanism underlying MFEs cannot easily be captured by current population dynamics models, this phenomena should not be ignored during analysis; there is a growing body of evidence that such collaborative activity may be a key towards unlocking the possible functional properties of many neuronal networks.