Biological complexity facilitates tuning of the neuronal parameter space.

The electrical and computational properties of neurons in our brains are determined by a rich repertoire of membrane-spanning ion channels and elaborate dendritic trees. However, the precise reason for this inherent complexity remains unknown, given that simpler models with fewer ion channels are also able to functionally reproduce the behaviour of some neurons. Here, we stochastically varied the ion channel densities of a biophysically detailed dentate gyrus granule cell model to produce a large population of putative granule cells, comparing those with all 15 original ion channels to their reduced but functional counterparts containing only 5 ion channels. Strikingly, valid parameter combinations in the full models were dramatically more frequent at ~6% vs. ~1% in the simpler model. The full models were also more stable in the face of perturbations to channel expression levels. Scaling up the numbers of ion channels artificially in the reduced models recovered these advantages confirming the key contribution of the actual number of ion channel types. We conclude that the diversity of ion channels gives a neuron greater flexibility and robustness to achieve a target excitability.

Synaptic logistics: Competing over shared resources.

High turnover rates of synaptic proteins imply that synapses constantly need to replace their constituent building blocks. This requires sophisticated supply chains and potentially exposes synapses to shortages as they compete for limited resources. Interestingly, competition in neurons has been observed at different scales. Whether it is competition of receptors for binding sites inside a single synapse or synapses fighting for resources to grow. Here we review the implications of such competition for synaptic function and plasticity. We identify multiple mechanisms that synapses use to safeguard themselves against supply shortages and identify a fundamental neurologistic trade-off governing the sizes of reserve pools of essential synaptic building blocks.

Active efficient coding explains the development of binocular vision and its failure in amblyopia.

The development of vision during the first months of life is an active process that comprises the learning of appropriate neural representations and the learning of accurate eye movements. While it has long been suspected that the two learning processes are coupled, there is still no widely accepted theoretical framework describing this joint development. Here, we propose a computational model of the development of active binocular vision to fill this gap. The model is based on a formulation of the active efficient coding theory, which proposes that eye movements as well as stimulus encoding are jointly adapted to maximize the overall coding efficiency. Under healthy conditions, the model self-calibrates to perform accurate vergence and accommodation eye movements. It exploits disparity cues to deduce the direction of defocus, which leads to coordinated vergence and accommodation responses. In a simulated anisometropic case, where the refraction power of the two eyes differs, an amblyopia-like state develops in which the foveal region of one eye is suppressed due to inputs from the other eye. After correcting for refractive errors, the model can only reach healthy performance levels if receptive fields are still plastic, in line with findings on a critical period for binocular vision development. Overall, our model offers a unifying conceptual framework for understanding the development of binocular vision.

Models of vision need some action.

Bowers et al. focus their criticisms on research that compares behavioral and brain data from the ventral stream with a class of deep neural networks for object recognition. While they are right to identify issues with current benchmarking research programs, they overlook a much more fundamental limitation of this literature: Disregarding the importance of action and interaction for perception.

Active head rolls enhance sonar-based auditory localization performance.

Animals utilize a variety of active sensing mechanisms to perceive the world around them. Echolocating bats are an excellent model for the study of active auditory localization. The big brown bat (Eptesicus fuscus), for instance, employs active head roll movements during sonar prey tracking. The function of head rolls in sound source localization is not well understood. Here, we propose an echolocation model with multi-axis head rotation to investigate the effect of active head roll movements on sound localization performance. The model autonomously learns to align the bat's head direction towards the target. We show that a model with active head roll movements better localizes targets than a model without head rolls. Furthermore, we demonstrate that active head rolls also reduce the time required for localization in elevation. Finally, our model offers key insights to sound localization cues used by echolocating bats employing active head movements during echolocation.

A Model of Brain Folding Based on Strong Local and Weak Long-Range Connectivity Requirements.

Throughout the animal kingdom, the structure of the central nervous system varies widely from distributed ganglia in worms to compact brains with varying degrees of folding in mammals. The differences in structure may indicate a fundamentally different circuit organization. However, the folded brain most likely is a direct result of mechanical forces when considering that a larger surface area of cortex packs into the restricted volume provided by the skull. Here, we introduce a computational model that instead of modeling mechanical forces relies on dimension reduction methods to place neurons according to specific connectivity requirements. For a simplified connectivity with strong local and weak long-range connections, our model predicts a transition from separate ganglia through smooth brain structures to heavily folded brains as the number of cortical columns increases. The model reproduces experimentally determined relationships between metrics of cortical folding and its pathological phenotypes in lissencephaly, polymicrogyria, microcephaly, autism, and schizophrenia. This suggests that mechanical forces that are known to lead to cortical folding may synergistically contribute to arrangements that reduce wiring. Our model provides a unified conceptual understanding of gyrification linking cellular connectivity and macroscopic structures in large-scale neural network models of the brain.

Recurrent processing improves occluded object recognition and gives rise to perceptual hysteresis.

Over the past decades, object recognition has been predominantly studied and modelled as a feedforward process. This notion was supported by the fast response times in psychophysical and neurophysiological experiments and the recent success of deep feedforward neural networks for object recognition. Recently, however, this prevalent view has shifted and recurrent connectivity in the brain is now believed to contribute significantly to object recognition - especially under challenging conditions, including the recognition of partially occluded objects. Moreover, recurrent dynamics might be the key to understanding perceptual phenomena such as perceptual hysteresis. In this work we investigate if and how artificial neural networks can benefit from recurrent connections. We systematically compare architectures comprised of bottom-up, lateral, and top-down connections. To evaluate the impact of recurrent connections for occluded object recognition, we introduce three stereoscopic occluded object datasets, which span the range from classifying partially occluded hand-written digits to recognizing three-dimensional objects. We find that recurrent architectures perform significantly better than parameter-matched feedforward models. An analysis of the hidden representation of the models suggests that occluders are progressively discounted in later time steps of processing. We demonstrate that feedback can correct the initial misclassifications over time and that the recurrent dynamics lead to perceptual hysteresis. Overall, our results emphasize the importance of recurrent feedback for object recognition in difficult situations.

Bridging structure and function: A model of sequence learning and prediction in primary visual cortex.

Recent experiments have demonstrated that visual cortex engages in spatio-temporal sequence learning and prediction. The cellular basis of this learning remains unclear, however. Here we present a spiking neural network model that explains a recent study on sequence learning in the primary visual cortex of rats. The model posits that the sequence learning and prediction abilities of cortical circuits result from the interaction of spike-timing dependent plasticity (STDP) and homeostatic plasticity mechanisms. It also reproduces changes in stimulus-evoked multi-unit activity during learning. Furthermore, it makes precise predictions regarding how training shapes network connectivity to establish its prediction ability. Finally, it predicts that the adapted connectivity gives rise to systematic changes in spontaneous network activity. Taken together, our model establishes a new conceptual bridge between the structure and function of cortical circuits in the context of sequence learning and prediction.

Simulation of electromyographic recordings following transcranial magnetic stimulation.

Transcranial magnetic stimulation (TMS) is a technique that enables noninvasive manipulation of neural activity and holds promise in both clinical and basic research settings. The effect of TMS on the motor cortex is often measured by electromyography (EMG) recordings from a small hand muscle. However, the details of how TMS generates responses measured with EMG are not completely understood. We aim to develop a biophysically detailed computational model to study the potential mechanisms underlying the generation of EMG signals following TMS. Our model comprises a feed-forward network of cortical layer 2/3 cells, which drive morphologically detailed layer 5 corticomotoneuronal cells, which in turn project to a pool of motoneurons. EMG signals are modeled as the sum of motor unit action potentials. EMG recordings from the first dorsal interosseous muscle were performed in four subjects and compared with simulated EMG signals. Our model successfully reproduces several characteristics of the experimental data. The simulated EMG signals match experimental EMG recordings in shape and size, and change with stimulus intensity and contraction level as in experimental recordings. They exhibit cortical silent periods that are close to the biological values and reveal an interesting dependence on inhibitory synaptic transmission properties. Our model predicts several characteristics of the firing patterns of neurons along the entire pathway from cortical layer 2/3 cells down to spinal motoneurons and should be considered as a viable tool for explaining and analyzing EMG signals following TMS. NEW & NOTEWORTHY A biophysically detailed model of EMG signal generation following transcranial magnetic stimulation (TMS) is proposed. Simulated EMG signals match experimental EMG recordings in shape and amplitude. Motor-evoked potential and cortical silent period properties match experimental data. The model is a viable tool to analyze, explain, and predict EMG signals following TMS.

A model of human motor sequence learning explains facilitation and interference effects based on spike-timing dependent plasticity.

The ability to learn sequential behaviors is a fundamental property of our brains. Yet a long stream of studies including recent experiments investigating motor sequence learning in adult human subjects have produced a number of puzzling and seemingly contradictory results. In particular, when subjects have to learn multiple action sequences, learning is sometimes impaired by proactive and retroactive interference effects. In other situations, however, learning is accelerated as reflected in facilitation and transfer effects. At present it is unclear what the underlying neural mechanism are that give rise to these diverse findings. Here we show that a recently developed recurrent neural network model readily reproduces this diverse set of findings. The self-organizing recurrent neural network (SORN) model is a network of recurrently connected threshold units that combines a simplified form of spike-timing dependent plasticity (STDP) with homeostatic plasticity mechanisms ensuring network stability, namely intrinsic plasticity (IP) and synaptic normalization (SN). When trained on sequence learning tasks modeled after recent experiments we find that it reproduces the full range of interference, facilitation, and transfer effects. We show how these effects are rooted in the network's changing internal representation of the different sequences across learning and how they depend on an interaction of training schedule and task similarity. Furthermore, since learning in the model is based on fundamental neuronal plasticity mechanisms, the model reveals how these plasticity mechanisms are ultimately responsible for the network's sequence learning abilities. In particular, we find that all three plasticity mechanisms are essential for the network to learn effective internal models of the different training sequences. This ability to form effective internal models is also the basis for the observed interference and facilitation effects. This suggests that STDP, IP, and SN may be the driving forces behind our ability to learn complex action sequences.

Plasticity-Driven Self-Organization under Topological Constraints Accounts for Non-random Features of Cortical Synaptic Wiring.

Understanding the structure and dynamics of cortical connectivity is vital to understanding cortical function. Experimental data strongly suggest that local recurrent connectivity in the cortex is significantly non-random, exhibiting, for example, above-chance bidirectionality and an overrepresentation of certain triangular motifs. Additional evidence suggests a significant distance dependency to connectivity over a local scale of a few hundred microns, and particular patterns of synaptic turnover dynamics, including a heavy-tailed distribution of synaptic efficacies, a power law distribution of synaptic lifetimes, and a tendency for stronger synapses to be more stable over time. Understanding how many of these non-random features simultaneously arise would provide valuable insights into the development and function of the cortex. While previous work has modeled some of the individual features of local cortical wiring, there is no model that begins to comprehensively account for all of them. We present a spiking network model of a rodent Layer 5 cortical slice which, via the interactions of a few simple biologically motivated intrinsic, synaptic, and structural plasticity mechanisms, qualitatively reproduces these non-random effects when combined with simple topological constraints. Our model suggests that mechanisms of self-organization arising from a small number of plasticity rules provide a parsimonious explanation for numerous experimentally observed non-random features of recurrent cortical wiring. Interestingly, similar mechanisms have been shown to endow recurrent networks with powerful learning abilities, suggesting that these mechanism are central to understanding both structure and function of cortical synaptic wiring.

Personalized translational epilepsy research - Novel approaches and future perspectives: Part II: Experimental and translational approaches.

Despite the availability of more than 15 new "antiepileptic drugs", the proportion of patients with pharmacoresistant epilepsy has remained constant at about 20-30%. Furthermore, no disease-modifying treatments shown to prevent the development of epilepsy following an initial precipitating brain injury or to reverse established epilepsy have been identified to date. This is likely in part due to the polyetiologic nature of epilepsy, which in turn requires personalized medicine approaches. Recent advances in imaging, pathology, genetics, and epigenetics have led to new pathophysiological concepts and the identification of monogenic causes of epilepsy. In the context of these advances, the First International Symposium on Personalized Translational Epilepsy Research (1st ISymPTER) was held in Frankfurt on September 8, 2016, to discuss novel approaches and future perspectives for personalized translational research. These included new developments and ideas in a range of experimental and clinical areas such as deep phenotyping, quantitative brain imaging, EEG/MEG-based analysis of network dysfunction, tissue-based translational studies, innate immunity mechanisms, microRNA as treatment targets, functional characterization of genetic variants in human cell models and rodent organotypic slice cultures, personalized treatment approaches for monogenic epilepsies, blood-brain barrier dysfunction, therapeutic focal tissue modification, computational modeling for target and biomarker identification, and cost analysis in (monogenic) disease and its treatment. This report on the meeting proceedings is aimed at stimulating much needed investments of time and resources in personalized translational epilepsy research. This Part II includes the experimental and translational approaches and a discussion of the future perspectives, while the diagnostic methods, EEG network analysis, biomarkers, and personalized treatment approaches were addressed in Part I [1].

Personalized translational epilepsy research - Novel approaches and future perspectives: Part I: Clinical and network analysis approaches.

Despite the availability of more than 15 new "antiepileptic drugs", the proportion of patients with pharmacoresistant epilepsy has remained constant at about 20-30%. Furthermore, no disease-modifying treatments shown to prevent the development of epilepsy following an initial precipitating brain injury or to reverse established epilepsy have been identified to date. This is likely in part due to the polyetiologic nature of epilepsy, which in turn requires personalized medicine approaches. Recent advances in imaging, pathology, genetics and epigenetics have led to new pathophysiological concepts and the identification of monogenic causes of epilepsy. In the context of these advances, the First International Symposium on Personalized Translational Epilepsy Research (1st ISymPTER) was held in Frankfurt on September 8, 2016, to discuss novel approaches and future perspectives for personalized translational research. These included new developments and ideas in a range of experimental and clinical areas such as deep phenotyping, quantitative brain imaging, EEG/MEG-based analysis of network dysfunction, tissue-based translational studies, innate immunity mechanisms, microRNA as treatment targets, functional characterization of genetic variants in human cell models and rodent organotypic slice cultures, personalized treatment approaches for monogenic epilepsies, blood-brain barrier dysfunction, therapeutic focal tissue modification, computational modeling for target and biomarker identification, and cost analysis in (monogenic) disease and its treatment. This report on the meeting proceedings is aimed at stimulating much needed investments of time and resources in personalized translational epilepsy research. Part I includes the clinical phenotyping and diagnostic methods, EEG network-analysis, biomarkers, and personalized treatment approaches. In Part II, experimental and translational approaches will be discussed (Bauer et al., 2017) [1].

Where's the Noise? Key Features of Spontaneous Activity and Neural Variability Arise through Learning in a Deterministic Network.

Even in the absence of sensory stimulation the brain is spontaneously active. This background "noise" seems to be the dominant cause of the notoriously high trial-to-trial variability of neural recordings. Recent experimental observations have extended our knowledge of trial-to-trial variability and spontaneous activity in several directions: 1. Trial-to-trial variability systematically decreases following the onset of a sensory stimulus or the start of a motor act. 2. Spontaneous activity states in sensory cortex outline the region of evoked sensory responses. 3. Across development, spontaneous activity aligns itself with typical evoked activity patterns. 4. The spontaneous brain activity prior to the presentation of an ambiguous stimulus predicts how the stimulus will be interpreted. At present it is unclear how these observations relate to each other and how they arise in cortical circuits. Here we demonstrate that all of these phenomena can be accounted for by a deterministic self-organizing recurrent neural network model (SORN), which learns a predictive model of its sensory environment. The SORN comprises recurrently coupled populations of excitatory and inhibitory threshold units and learns via a combination of spike-timing dependent plasticity (STDP) and homeostatic plasticity mechanisms. Similar to balanced network architectures, units in the network show irregular activity and variable responses to inputs. Additionally, however, the SORN exhibits sequence learning abilities matching recent findings from visual cortex and the network's spontaneous activity reproduces the experimental findings mentioned above. Intriguingly, the network's behaviour is reminiscent of sampling-based probabilistic inference, suggesting that correlates of sampling-based inference can develop from the interaction of STDP and homeostasis in deterministic networks. We conclude that key observations on spontaneous brain activity and the variability of neural responses can be accounted for by a simple deterministic recurrent neural network which learns a predictive model of its sensory environment via a combination of generic neural plasticity mechanisms.

An active-efficient-coding model of optokinetic nystagmus.

Optokinetic nystagmus (OKN) is an involuntary eye movement responsible for stabilizing retinal images in the presence of relative motion between an observer and the environment. Fully understanding the development of OKN requires a neurally plausible computational model that accounts for the neural development and the behavior. To date, work in this area has been limited. We propose a neurally plausible framework for the joint development of disparity and motion tuning in the visual cortex and of optokinetic and vergence eye-movement behavior. To our knowledge, this framework is the first developmental model to describe the emergence of OKN in a behaving organism. Unlike past models, which were based on scalar models of overall activity in different neural areas, our framework models the development of the detailed connectivity both from the retinal input to the visual cortex and from the visual cortex to the motor neurons. This framework accounts for the importance of the development of normal vergence control and binocular vision in achieving normal monocular OKN behaviors. Because the model includes behavior, we can simulate the same perturbations as past experiments, such as artificially induced strabismus. The proposed model agrees both qualitatively and quantitatively with a number of findings from the literature on both binocular vision and the optokinetic reflex. Finally, our model makes quantitative predictions about OKN behavior using the same methods used to characterize OKN in the experimental literature.

OpenEyeSim: A biomechanical model for simulation of closed-loop visual perception.

We introduce OpenEyeSim, a detailed three-dimensional biomechanical model of the human extraocular eye muscles including a visualization of a virtual environment. The main purpose of OpenEyeSim is to serve as a platform for developing models of the joint learning of visual representations and eye-movement control in the perception-action cycle. The architecture and dynamic muscle properties are based on measurements of the human oculomotor system. We show that our model can reproduce different types of eye movements. Additionally, our model is able to calculate metabolic costs of eye movements. It is also able to simulate different eye disorders, such as different forms of strabismus. We propose OpenEyeSim as a platform for studying many of the complexities of oculomotor control and learning during normal and abnormal visual development.

Network self-organization explains the statistics and dynamics of synaptic connection strengths in cortex.

The information processing abilities of neural circuits arise from their synaptic connection patterns. Understanding the laws governing these connectivity patterns is essential for understanding brain function. The overall distribution of synaptic strengths of local excitatory connections in cortex and hippocampus is long-tailed, exhibiting a small number of synaptic connections of very large efficacy. At the same time, new synaptic connections are constantly being created and individual synaptic connection strengths show substantial fluctuations across time. It remains unclear through what mechanisms these properties of neural circuits arise and how they contribute to learning and memory. In this study we show that fundamental characteristics of excitatory synaptic connections in cortex and hippocampus can be explained as a consequence of self-organization in a recurrent network combining spike-timing-dependent plasticity (STDP), structural plasticity and different forms of homeostatic plasticity. In the network, associative synaptic plasticity in the form of STDP induces a rich-get-richer dynamics among synapses, while homeostatic mechanisms induce competition. Under distinctly different initial conditions, the ensuing self-organization produces long-tailed synaptic strength distributions matching experimental findings. We show that this self-organization can take place with a purely additive STDP mechanism and that multiplicative weight dynamics emerge as a consequence of network interactions. The observed patterns of fluctuation of synaptic strengths, including elimination and generation of synaptic connections and long-term persistence of strong connections, are consistent with the dynamics of dendritic spines found in rat hippocampus. Beyond this, the model predicts an approximately power-law scaling of the lifetimes of newly established synaptic connection strengths during development. Our results suggest that the combined action of multiple forms of neuronal plasticity plays an essential role in the formation and maintenance of cortical circuits.

Watch the hands: infants can learn to follow gaze by seeing adults manipulate objects.

Infants gradually learn to share attention, but it is unknown how they acquire skills such as gaze-following. Deák and Triesch (2006) suggest that gaze-following could be acquired if infants learn that adults' gaze direction is likely to be aligned with interesting sights. This hypothesis stipulates that adults tend to look at things that infants find interesting, and that infants could learn by noticing this tendency. We tested the plausibility of this hypothesis through video-based micro-behavioral analysis of naturalistic parent-infant play. The results revealed that 3- to 11-month-old infants strongly preferred watching caregivers handle objects. In addition, when caregivers looked away from their infant they tended to look at their own object-handling. Finally, when infants looked toward the caregiver while she was looking at her own hands, the infant's next eye movement was often toward the caregiver's object-handling. In this way infants receive adequate naturalistic input to learn associations between their parent's gaze direction and the locations of interesting sights.

Differential effects of visual attention and working memory on binocular rivalry.

The investigation of cognitive influence on binocular rivalry has a long history. However, the effects of visual WM on rivalry have never been studied so far. We examined top-down modulation of rivalry perception in four experiments to compare the effects of visual WM and sustained selective attention: In the first three experiments we failed to observe any sustained effect of the WM content; only the color of the memory probe was found to prime the initially dominant percept. In Experiment 4 we found a clear effect of sustained attention on rivalry both in terms of the first dominant percept and of the overall dominance when participants were involved in a tracking task. Our results provide an example of dissociation between visual WM and selective attention, two phenomena which otherwise functionally overlap to a large extent. Furthermore, our study highlights the importance of the task employed to engage cognitive resources: The observed perceptual epiphenomena of binocular rivalry are indicative of visual competition at an early stage, which is not affected by WM but is still susceptible to attention influence as long as the observer's attention is constrained to one of the two rival images via a specific concomitant task.

Accelerating spiking neural network simulations with PymoNNto and PymoNNtorch.

Spiking neural network simulations are a central tool in Computational Neuroscience, Artificial Intelligence, and Neuromorphic Engineering research. A broad range of simulators and software frameworks for such simulations exist with different target application areas. Among these, PymoNNto is a recent Python-based toolbox for spiking neural network simulations that emphasizes the embedding of custom code in a modular and flexible way. While PymoNNto already supports GPU implementations, its backend relies on NumPy operations. Here we introduce PymoNNtorch, which is natively implemented with PyTorch while retaining PymoNNto's modular design. Furthermore, we demonstrate how changes to the implementations of common network operations in combination with PymoNNtorch's native GPU support can offer speed-up over conventional simulators like NEST, ANNarchy, and Brian 2 in certain situations. Overall, we show how PymoNNto's modular and flexible design in combination with PymoNNtorch's GPU acceleration and optimized indexing operations facilitate research and development of spiking neural networks in the Python programming language.