Constructive connectomics: How neuronal axons get from here to there using gene-expression maps derived from their family trees.

During brain development, billions of axons must navigate over multiple spatial scales to reach specific neuronal targets, and so build the processing circuits that generate the intelligent behavior of animals. However, the limited information capacity of the zygotic genome puts a strong constraint on how, and which, axonal routes can be encoded. We propose and validate a mechanism of development that can provide an efficient encoding of this global wiring task. The key principle, confirmed through simulation, is that basic constraints on mitoses of neural stem cells-that mitotic daughters have similar gene expression to their parent and do not stray far from one another-induce a global hierarchical map of nested regions, each marked by the expression profile of its common progenitor population. Thus, a traversal of the lineal hierarchy generates a systematic sequence of expression profiles that traces a staged route, which growth cones can follow to their remote targets. We have analyzed gene expression data of developing and adult mouse brains published by the Allen Institute for Brain Science, and found them consistent with our simulations: gene expression indeed partitions the brain into a global spatial hierarchy of nested contiguous regions that is stable at least from embryonic day 11.5 to postnatal day 56. We use this experimental data to demonstrate that our axonal guidance algorithm is able to robustly extend arbors over long distances to specific targets, and that these connections result in a qualitatively plausible connectome. We conclude that, paradoxically, cell division may be the key to uniting the neurons of the brain.

Conditioning by subthreshold synaptic input changes the intrinsic firing pattern of CA3 hippocampal neurons.

Unlike synaptic strength, intrinsic excitability is assumed to be a stable property of neurons. For example, learning of somatic conductances is generally not incorporated into computational models, and the discharge pattern of neurons in response to test stimuli is frequently used as a basis for phenotypic classification. However, it is increasingly evident that signal processing properties of neurons are more generally plastic on the timescale of minutes. Here we demonstrate that the intrinsic firing patterns of CA3 neurons of the rat hippocampus in vitro undergo rapid long-term plasticity in response to a few minutes of only subthreshold synaptic conditioning. This plasticity on the spike timing could also be induced by intrasomatic injection of subthreshold depolarizing pulses and was blocked by kinase inhibitors, indicating that discharge dynamics are modulated locally. Cluster analysis of firing patterns before and after conditioning revealed systematic transitions toward adapting and intrinsic burst behaviors, irrespective of the patterns initially exhibited by the cells. We used a conductance-based model to decide appropriate pharmacological blockade and found that the observed transitions are likely due to recruitment of low-voltage calcium and Kv7 potassium conductances. We conclude that CA3 neurons adapt their conductance profile to the subthreshold activity of their input, so that their intrinsic firing pattern is not a static signature, but rather a reflection of their history of subthreshold activity. In this way, recurrent output from CA3 neurons may collectively shape the temporal dynamics of their embedding circuits.NEW & NOTEWORTHY Although firing patterns are widely conserved across the animal phyla, it is still a mystery why nerve cells present such diversity of discharge dynamics upon somatic step currents. Adding a new timing dimension to the intrinsic plasticity literature, here we show that CA3 neurons rapidly adapt through the space of known firing patterns in response to the subthreshold signals that they receive from their embedding circuit, potentially adjusting their network processing to the temporal statistics of their circuit.

Solving Constraint-Satisfaction Problems with Distributed Neocortical-Like Neuronal Networks.

Finding actions that satisfy the constraints imposed by both external inputs and internal representations is central to decision making. We demonstrate that some important classes of constraint satisfaction problems (CSPs) can be solved by networks composed of homogeneous cooperative-competitive modules that have connectivity similar to motifs observed in the superficial layers of neocortex. The winner-take-all modules are sparsely coupled by programming neurons that embed the constraints onto the otherwise homogeneous modular computational substrate. We show rules that embed any instance of the CSP's planar four-color graph coloring, maximum independent set, and sudoku on this substrate and provide mathematical proofs that guarantee these graph coloring problems will convergence to a solution. The network is composed of nonsaturating linear threshold neurons. Their lack of right saturation allows the overall network to explore the problem space driven through the unstable dynamics generated by recurrent excitation. The direction of exploration is steered by the constraint neurons. While many problems can be solved using only linear inhibitory constraints, network performance on hard problems benefits significantly when these negative constraints are implemented by nonlinear multiplicative inhibition. Overall, our results demonstrate the importance of instability rather than stability in network computation and offer insight into the computational role of dual inhibitory mechanisms in neural circuits.

Unsupervised lineage-based characterization of primate precursors reveals high proliferative and morphological diversity in the OSVZ.

Generation of the primate cortex is characterized by the diversity of cortical precursors and the complexity of their lineage relationships. Recent studies have reported miscellaneous precursor types based on observer classification of cell biology features including morphology, stemness, and proliferative behavior. Here we use an unsupervised machine learning method for Hidden Markov Trees (HMTs), which can be applied to large datasets to classify precursors on the basis of morphology, cell-cycle length, and behavior during mitosis. The unbiased lineage analysis automatically identifies cell types by applying a lineage-based clustering and model-learning algorithm to a macaque corticogenesis dataset. The algorithmic results validate previously reported observer classification of precursor types and show numerous advantages: It predicts a higher diversity of progenitors and numerous potential transitions between precursor types. The HMT model can be initialized to learn a user-defined number of distinct classes of precursors. This makes it possible to 1) reveal as yet undetected precursor types in view of exploring the significant features of precursors with respect to specific cellular processes; and 2) explore specific lineage features. For example, most precursors in the experimental dataset exhibit bidirectional transitions. Constraining the directionality in the HMT model leads to a reduction in precursor diversity following multiple divisions, thereby suggesting that one impact of bidirectionality in corticogenesis is to maintain precursor diversity. In this way we show that unsupervised lineage analysis provides a valuable methodology for investigating fundamental features of corticogenesis.

Computation in dynamically bounded asymmetric systems.

Previous explanations of computations performed by recurrent networks have focused on symmetrically connected saturating neurons and their convergence toward attractors. Here we analyze the behavior of asymmetrical connected networks of linear threshold neurons, whose positive response is unbounded. We show that, for a wide range of parameters, this asymmetry brings interesting and computationally useful dynamical properties. When driven by input, the network explores potential solutions through highly unstable 'expansion' dynamics. This expansion is steered and constrained by negative divergence of the dynamics, which ensures that the dimensionality of the solution space continues to reduce until an acceptable solution manifold is reached. Then the system contracts stably on this manifold towards its final solution trajectory. The unstable positive feedback and cross inhibition that underlie expansion and divergence are common motifs in molecular and neuronal networks. Therefore we propose that very simple organizational constraints that combine these motifs can lead to spontaneous computation and so to the spontaneous modification of entropy that is characteristic of living systems.

Developmental self-construction and -configuration of functional neocortical neuronal networks.

The prenatal development of neural circuits must provide sufficient configuration to support at least a set of core postnatal behaviors. Although knowledge of various genetic and cellular aspects of development is accumulating rapidly, there is less systematic understanding of how these various processes play together in order to construct such functional networks. Here we make some steps toward such understanding by demonstrating through detailed simulations how a competitive co-operative ('winner-take-all', WTA) network architecture can arise by development from a single precursor cell. This precursor is granted a simplified gene regulatory network that directs cell mitosis, differentiation, migration, neurite outgrowth and synaptogenesis. Once initial axonal connection patterns are established, their synaptic weights undergo homeostatic unsupervised learning that is shaped by wave-like input patterns. We demonstrate how this autonomous genetically directed developmental sequence can give rise to self-calibrated WTA networks, and compare our simulation results with biological data.

Developmental origin of patchy axonal connectivity in the neocortex: a computational model.

Injections of neural tracers into many mammalian neocortical areas reveal a common patchy motif of clustered axonal projections. We studied in simulation a mathematical model for neuronal development in order to investigate how this patchy connectivity could arise in layer II/III of the neocortex. In our model, individual neurons of this layer expressed the activator-inhibitor components of a Gierer-Meinhardt reaction-diffusion system. The resultant steady-state reaction-diffusion pattern across the neuronal population was approximately hexagonal. Growth cones at the tips of extending axons used the various morphogens secreted by intrapatch neurons as guidance cues to direct their growth and invoke axonal arborization, so yielding a patchy distribution of arborization across the entire layer II/III. We found that adjustment of a single parameter yields the intriguing linear relationship between average patch diameter and interpatch spacing that has been observed experimentally over many cortical areas and species. We conclude that a simple Gierer-Meinhardt system expressed by the neurons of the developing neocortex is sufficient to explain the patterns of clustered connectivity observed experimentally.

Precursor diversity and complexity of lineage relationships in the outer subventricular zone of the primate.

Long-term ex vivo live imaging combined with unbiased sampling of cycling precursors shows that macaque outer subventricular zone (OSVZ) includes four distinct basal radial glial (bRG) cell morphotypes, bearing apical and/or basal processes in addition to nonpolar intermediate progenitors (IPs). Each of the five precursor types exhibits extensive self-renewal and proliferative capacities as well as the ability to directly generate neurons, albeit with different frequencies. Cell-cycle parameters exhibited an unusual stage-specific regulation with short cell-cycle duration and increased rates of proliferative divisions during supragranular layer production at late corticogenesis. State transition analysis of an extensive clonal database reveals bidirectional transitions between OSVZ precursor types as well as stage-specific differences in their progeny and topology of the lineage relationships. These results explore rodent-primate differences and show that primate cortical neurons are generated through complex lineages by a mosaic of precursors, thereby providing an innovative framework for understanding specific features of primate corticogenesis.

Simulating cortical development as a self constructing process: a novel multi-scale approach combining molecular and physical aspects.

Current models of embryological development focus on intracellular processes such as gene expression and protein networks, rather than on the complex relationship between subcellular processes and the collective cellular organization these processes support. We have explored this collective behavior in the context of neocortical development, by modeling the expansion of a small number of progenitor cells into a laminated cortex with layer and cell type specific projections. The developmental process is steered by a formal language analogous to genomic instructions, and takes place in a physically realistic three-dimensional environment. A common genome inserted into individual cells control their individual behaviors, and thereby gives rise to collective developmental sequences in a biologically plausible manner. The simulation begins with a single progenitor cell containing the artificial genome. This progenitor then gives rise through a lineage of offspring to distinct populations of neuronal precursors that migrate to form the cortical laminae. The precursors differentiate by extending dendrites and axons, which reproduce the experimentally determined branching patterns of a number of different neuronal cell types observed in the cat visual cortex. This result is the first comprehensive demonstration of the principles of self-construction whereby the cortical architecture develops. In addition, our model makes several testable predictions concerning cell migration and branching mechanisms.

Synthesizing cognition in neuromorphic electronic systems.

The quest to implement intelligent processing in electronic neuromorphic systems lacks methods for achieving reliable behavioral dynamics on substrates of inherently imprecise and noisy neurons. Here we report a solution to this problem that involves first mapping an unreliable hardware layer of spiking silicon neurons into an abstract computational layer composed of generic reliable subnetworks of model neurons and then composing the target behavioral dynamics as a "soft state machine" running on these reliable subnets. In the first step, the neural networks of the abstract layer are realized on the hardware substrate by mapping the neuron circuit bias voltages to the model parameters. This mapping is obtained by an automatic method in which the electronic circuit biases are calibrated against the model parameters by a series of population activity measurements. The abstract computational layer is formed by configuring neural networks as generic soft winner-take-all subnetworks that provide reliable processing by virtue of their active gain, signal restoration, and multistability. The necessary states and transitions of the desired high-level behavior are then easily embedded in the computational layer by introducing only sparse connections between some neurons of the various subnets. We demonstrate this synthesis method for a neuromorphic sensory agent that performs real-time context-dependent classification of motion patterns observed by a silicon retina.

Spike-based probabilistic inference in analog graphical models using interspike-interval coding.

Temporal spike codes play a crucial role in neural information processing. In particular, there is strong experimental evidence that interspike intervals (ISIs) are used for stimulus representation in neural systems. However, very few algorithmic principles exploit the benefits of such temporal codes for probabilistic inference of stimuli or decisions. Here, we describe and rigorously prove the functional properties of a spike-based processor that uses ISI distributions to perform probabilistic inference. The abstract processor architecture serves as a building block for more concrete, neural implementations of the belief-propagation (BP) algorithm in arbitrary graphical models (e.g., Bayesian networks and factor graphs). The distributed nature of graphical models matches well with the architectural and functional constraints imposed by biology. In our model, ISI distributions represent the BP messages exchanged between factor nodes, leading to the interpretation of a single spike as a random sample that follows such a distribution. We verify the abstract processor model by numerical simulation in full graphs, and demonstrate that it can be applied even in the presence of analog variables. As a particular example, we also show results of a concrete, neural implementation of the processor, although in principle our approach is more flexible and allows different neurobiological interpretations. Furthermore, electrophysiological data from area LIP during behavioral experiments are assessed in light of ISI coding, leading to concrete testable, quantitative predictions and a more accurate description of these data compared to hitherto existing models.

TrakEM2 software for neural circuit reconstruction.

A key challenge in neuroscience is the expeditious reconstruction of neuronal circuits. For model systems such as Drosophila and C. elegans, the limiting step is no longer the acquisition of imagery but the extraction of the circuit from images. For this purpose, we designed a software application, TrakEM2, that addresses the systematic reconstruction of neuronal circuits from large electron microscopical and optical image volumes. We address the challenges of image volume composition from individual, deformed images; of the reconstruction of neuronal arbors and annotation of synapses with fast manual and semi-automatic methods; and the management of large collections of both images and annotations. The output is a neural circuit of 3d arbors and synapses, encoded in NeuroML and other formats, ready for analysis.

Competition through selective inhibitory synchrony.

Models of cortical neuronal circuits commonly depend on inhibitory feedback to control gain, provide signal normalization, and selectively amplify signals using winner-take-all (WTA) dynamics. Such models generally assume that excitatory and inhibitory neurons are able to interact easily because their axons and dendrites are colocalized in the same small volume. However, quantitative neuroanatomical studies of the dimensions of axonal and dendritic trees of neurons in the neocortex show that this colocalization assumption is not valid. In this letter, we describe a simple modification to the WTA circuit design that permits the effects of distributed inhibitory neurons to be coupled through synchronization, and so allows a single WTA to be distributed widely in cortical space, well beyond the arborization of any single inhibitory neuron and even across different cortical areas. We prove by nonlinear contraction analysis and demonstrate by simulation that distributed WTA subsystems combined by such inhibitory synchrony are inherently stable. We show analytically that synchronization is substantially faster than winner selection. This circuit mechanism allows networks of independent WTAs to fully or partially compete with other.

An instruction language for self-construction in the context of neural networks.

Biological systems are based on an entirely different concept of construction than human artifacts. They construct themselves by a process of self-organization that is a systematic spatio-temporal generation of, and interaction between, various specialized cell types. We propose a framework for designing gene-like codes for guiding the self-construction of neural networks. The description of neural development is formalized by defining a set of primitive actions taken locally by neural precursors during corticogenesis. These primitives can be combined into networks of instructions similar to biochemical pathways, capable of reproducing complex developmental sequences in a biologically plausible way. Moreover, the conditional activation and deactivation of these instruction networks can also be controlled by these primitives, allowing for the design of a "genetic code" containing both coding and regulating elements. We demonstrate in a simulation of physical cell development how this code can be incorporated into a single progenitor, which then by replication and differentiation, reproduces important aspects of corticogenesis.

A systematic method for configuring VLSI networks of spiking neurons.

An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electronic emulation conforms to a higher-level neuronal simulation. We show that the neurons configured by our method exhibit spike timing statistics and temporal dynamics that are the same as those observed in the software simulated neurons and, in particular, that the key parameters of recurrent VLSI neural networks (e.g., implementing soft winner-take-all) can be precisely tuned. The proposed method permits a seamless integration between software simulations with hardware emulations and intertranslatability between the parameters of abstract neuronal models and their emulation counterparts. Most important, our method offers a route toward a high-level task configuration language for neuromorphic VLSI systems.

Embedding of cortical representations by the superficial patch system.

Pyramidal cells in layers 2 and 3 of the neocortex of many species collectively form a clustered system of lateral axonal projections (the superficial patch system--Lund JS, Angelucci A, Bressloff PC. 2003. Anatomical substrates for functional columns in macaque monkey primary visual cortex. Cereb Cortex. 13:15-24. or daisy architecture--Douglas RJ, Martin KAC. 2004. Neuronal circuits of the neocortex. Annu Rev Neurosci. 27:419-451.), but the function performed by this general feature of the cortical architecture remains obscure. By comparing the spatial configuration of labeled patches with the configuration of responses to drifting grating stimuli, we found the spatial organizations both of the patch system and of the cortical response to be highly conserved between cat and monkey primary visual cortex. More importantly, the configuration of the superficial patch system is directly reflected in the arrangement of function across monkey primary visual cortex. Our results indicate a close relationship between the structure of the superficial patch system and cortical responses encoding a single value across the surface of visual cortex (self-consistent states). This relationship is consistent with the spontaneous emergence of orientation response-like activity patterns during ongoing cortical activity (Kenet T, Bibitchkov D, Tsodyks M, Grinvald A, Arieli A. 2003. Spontaneously emerging cortical representations of visual attributes. Nature. 425:954-956.). We conclude that the superficial patch system is the physical encoding of self-consistent cortical states, and that a set of concurrently labeled patches participate in a network of mutually consistent representations of cortical input.

What's black and white about the grey matter?

In 1873 Camillo Golgi discovered his eponymous stain, which he called la reazione nera. By adding to it the concepts of the Neuron Doctrine and the Law of Dynamic Polarisation, Santiago Ramon y Cajal was able to link the individual Golgi-stained neurons he saw down his microscope into circuits. This was revolutionary and we have all followed Cajal's winning strategy for over a century. We are now on the verge of a new revolution, which offers the prize of a far more comprehensive description of neural circuits and their operation. The hope is that we will exploit the power of computer vision algorithms and modern molecular biological techniques to acquire rapidly reconstructions of single neurons and synaptic circuits, and to control the function of selected types of neurons. Only one item is now conspicuous by its absence: the 21st century equivalent of the concepts of the Neuron Doctrine and the Law of Dynamic Polarisation. Without their equivalent we will inevitably struggle to make sense of our 21st century observations within the 19th and 20th century conceptual framework we have inherited.

Collective stability of networks of winner-take-all circuits.

The neocortex has a remarkably uniform neuronal organization, suggesting that common principles of processing are employed throughout its extent. In particular, the patterns of connectivity observed in the superficial layers of the visual cortex are consistent with the recurrent excitation and inhibitory feedback required for cooperative-competitive circuits such as the soft winner-take-all (WTA). WTA circuits offer interesting computational properties such as selective amplification, signal restoration, and decision making. But these properties depend on the signal gain derived from positive feedback, and so there is a critical trade-off between providing feedback strong enough to support the sophisticated computations while maintaining overall circuit stability. The issue of stability is all the more intriguing when one considers that the WTAs are expected to be densely distributed through the superficial layers and that they are at least partially interconnected. We consider how to reason about stability in very large distributed networks of such circuits. We approach this problem by approximating the regular cortical architecture as many interconnected cooperative-competitive modules. We demonstrate that by properly understanding the behavior of this small computational module, one can reason over the stability and convergence of very large networks composed of these modules. We obtain parameter ranges in which the WTA circuit operates in a high-gain regime, is stable, and can be aggregated arbitrarily to form large, stable networks. We use nonlinear contraction theory to establish conditions for stability in the fully nonlinear case and verify these solutions using numerical simulations. The derived bounds allow modes of operation in which the WTA network is multistable and exhibits state-dependent persistent activities. Our approach is sufficiently general to reason systematically about the stability of any network, biological or technological, composed of networks of small modules that express competition through shared inhibition.