Responses to Pattern-Violating Visual Stimuli Evolve Differently Over Days in Somata and Distal Apical Dendrites.

Scientists have long conjectured that the neocortex learns patterns in sensory data to generate top-down predictions of upcoming stimuli. In line with this conjecture, different responses to pattern-matching vs pattern-violating visual stimuli have been observed in both spiking and somatic calcium imaging data. However, it remains unknown whether these pattern-violation signals are different between the distal apical dendrites, which are heavily targeted by top-down signals, and the somata, where bottom-up information is primarily integrated. Furthermore, it is unknown how responses to pattern-violating stimuli evolve over time as an animal gains more experience with them. Here, we address these unanswered questions by analyzing responses of individual somata and dendritic branches of layer 2/3 and layer 5 pyramidal neurons tracked over multiple days in primary visual cortex of awake, behaving female and male mice. We use sequences of Gabor patches with patterns in their orientations to create pattern-matching and pattern-violating stimuli, and two-photon calcium imaging to record neuronal responses. Many neurons in both layers show large differences between their responses to pattern-matching and pattern-violating stimuli. Interestingly, these responses evolve in opposite directions in the somata and distal apical dendrites, with somata becoming less sensitive to pattern-violating stimuli and distal apical dendrites more sensitive. These differences between the somata and distal apical dendrites may be important for hierarchical computation of sensory predictions and learning, since these two compartments tend to receive bottom-up and top-down information, respectively.

Backpropagation and the brain.

During learning, the brain modifies synapses to improve behaviour. In the cortex, synapses are embedded within multilayered networks, making it difficult to determine the effect of an individual synaptic modification on the behaviour of the system. The backpropagation algorithm solves this problem in deep artificial neural networks, but historically it has been viewed as biologically problematic. Nonetheless, recent developments in neuroscience and the successes of artificial neural networks have reinvigorated interest in whether backpropagation offers insights for understanding learning in the cortex. The backpropagation algorithm learns quickly by computing synaptic updates using feedback connections to deliver error signals. Although feedback connections are ubiquitous in the cortex, it is difficult to see how they could deliver the error signals required by strict formulations of backpropagation. Here we build on past and recent developments to argue that feedback connections may instead induce neural activities whose differences can be used to locally approximate these signals and hence drive effective learning in deep networks in the brain.

Deep Learning with Dynamic Spiking Neurons and Fixed Feedback Weights.

Recent work in computer science has shown the power of deep learning driven by the backpropagation algorithm in networks of artificial neurons. But real neurons in the brain are different from most of these artificial ones in at least three crucial ways: they emit spikes rather than graded outputs, their inputs and outputs are related dynamically rather than by piecewise-smooth functions, and they have no known way to coordinate arrays of synapses in separate forward and feedback pathways so that they change simultaneously and identically, as they do in backpropagation. Given these differences, it is unlikely that current deep learning algorithms can operate in the brain, but we that show these problems can be solved by two simple devices: learning rules can approximate dynamic input-output relations with piecewise-smooth functions, and a variation on the feedback alignment algorithm can train deep networks without having to coordinate forward and feedback synapses. Our results also show that deep spiking networks learn much better if each neuron computes an intracellular teaching signal that reflects that cell's nonlinearity. With this mechanism, networks of spiking neurons show useful learning in synapses at least nine layers upstream from the output cells and perform well compared to other spiking networks in the literature on the MNIST digit recognition task.

Primary motor cortex neurons classified in a postural task predict muscle activation patterns in a reaching task.

Primary motor cortex (M1) activity correlates with many motor variables, making it difficult to demonstrate how it participates in motor control. We developed a two-stage process to separate the process of classifying the motor field of M1 neurons from the process of predicting the spatiotemporal patterns of its motor field during reaching. We tested our approach with a neural network model that controlled a two-joint arm to show the statistical relationship between network connectivity and neural activity across different motor tasks. In rhesus monkeys, M1 neurons classified by this method showed preferred reaching directions similar to their associated muscle groups. Importantly, the neural population signals predicted the spatiotemporal dynamics of their associated muscle groups, although a subgroup of atypical neurons reversed their directional preference, suggesting a selective role in antagonist control. These results highlight that M1 provides important details on the spatiotemporal patterns of muscle activity during motor skills such as reaching.

Temporal evolution of both premotor and motor cortical tuning properties reflect changes in limb biomechanics.

A prevailing theory in the cortical control of limb movement posits that premotor cortex initiates a high-level motor plan that is transformed by the primary motor cortex (MI) into a low-level motor command to be executed. This theory implies that the premotor cortex is shielded from the motor periphery, and therefore, its activity should not represent the low-level features of movement. Contrary to this theory, we show that both dorsal (PMd) and ventral premotor (PMv) cortexes exhibit population-level tuning properties that reflect the biomechanical properties of the periphery similar to those observed in M1. We recorded single-unit activity from M1, PMd, and PMv and characterized their tuning properties while six rhesus macaques performed a reaching task in the horizontal plane. Each area exhibited a bimodal distribution of preferred directions during execution consistent with the known biomechanical anisotropies of the muscles and limb segments. Moreover, these distributions varied in orientation or shape from planning to execution. A network model shows that such population dynamics are linked to a change in biomechanics of the limb as the monkey begins to move, specifically to the state-dependent properties of muscles. We suggest that, like M1, neural populations in PMd and PMv are more directly linked with the motor periphery than previously thought.

Adapting to inversion of the visual field: a new twist on an old problem.

While sensorimotor adaptation to prisms that displace the visual field takes minutes, adapting to an inversion of the visual field takes weeks. In spite of a long history of the study, the basis of this profound difference remains poorly understood. Here, we describe the computational issue that underpins this phenomenon and presents experiments designed to explore the mechanisms involved. We show that displacements can be mastered without altering the updated rule used to adjust the motor commands. In contrast, inversions flip the sign of crucial variables called sensitivity derivatives-variables that capture how changes in motor commands affect task error and therefore require an update of the feedback learning rule itself. Models of sensorimotor learning that assume internal estimates of these variables are known and fixed predicted that when the sign of a sensitivity derivative is flipped, adaptations should become increasingly counterproductive. In contrast, models that relearn these derivatives predict that performance should initially worsen, but then improve smoothly and remain stable once the estimate of the new sensitivity derivative has been corrected. Here, we evaluated these predictions by looking at human performance on a set of pointing tasks with vision perturbed by displacing and inverting prisms. Our experimental data corroborate the classic observation that subjects reduce their motor errors under inverted vision. Subjects' accuracy initially worsened and then improved. However, improvement was jagged rather than smooth and performance remained unstable even after 8 days of continually inverted vision, suggesting that subjects improve via an unknown mechanism, perhaps a combination of cognitive and implicit strategies. These results offer a new perspective on classic work with inverted vision.

Responses of pyramidal cell somata and apical dendrites in mouse visual cortex over multiple days.

The apical dendrites of pyramidal neurons in sensory cortex receive primarily top-down signals from associative and motor regions, while cell bodies and nearby dendrites are heavily targeted by locally recurrent or bottom-up inputs from the sensory periphery. Based on these differences, a number of theories in computational neuroscience postulate a unique role for apical dendrites in learning. However, due to technical challenges in data collection, little data is available for comparing the responses of apical dendrites to cell bodies over multiple days. Here we present a dataset collected through the Allen Institute Mindscope's OpenScope program that addresses this need. This dataset comprises high-quality two-photon calcium imaging from the apical dendrites and the cell bodies of visual cortical pyramidal neurons, acquired over multiple days in awake, behaving mice that were presented with visual stimuli. Many of the cell bodies and dendrite segments were tracked over days, enabling analyses of how their responses change over time. This dataset allows neuroscientists to explore the differences between apical and somatic processing and plasticity.

Rotational dynamics in motor cortex are consistent with a feedback controller.

Recent studies have identified rotational dynamics in motor cortex (MC), which many assume arise from intrinsic connections in MC. However, behavioral and neurophysiological studies suggest that MC behaves like a feedback controller where continuous sensory feedback and interactions with other brain areas contribute substantially to MC processing. We investigated these apparently conflicting theories by building recurrent neural networks that controlled a model arm and received sensory feedback from the limb. Networks were trained to counteract perturbations to the limb and to reach toward spatial targets. Network activities and sensory feedback signals to the network exhibited rotational structure even when the recurrent connections were removed. Furthermore, neural recordings in monkeys performing similar tasks also exhibited rotational structure not only in MC but also in somatosensory cortex. Our results argue that rotational structure may also reflect dynamics throughout the voluntary motor system involved in online control of motor actions.

Complex spatiotemporal tuning in human upper-limb muscles.

Correlations between neural activity in primary motor cortex (M1) and arm kinematics have recently been shown to be temporally extensive and spatially complex. These results provide a sophisticated account of M1 processing and suggest that M1 neurons encode high-level movement trajectories, termed "pathlets." However, interpreting pathlets is difficult because the mapping between M1 activity and arm kinematics is indirect: M1 activity can generate movement only via spinal circuitry and the substantial complexities of the musculoskeletal system. We hypothesized that filter-like complexities of the musculoskeletal system are sufficient to generate temporally extensive and spatially complex correlations between motor commands and arm kinematics. To test this hypothesis, we extended the computational and experimental method proposed for extracting pathlets from M1 activity to extract pathlets from muscle activity. Unlike M1 activity, it is clear that muscle activity does not encode arm kinematics. Accordingly, any spatiotemporal correlations in muscle pathlets can be attributed to musculoskeletal complexities rather than explicit higher-order representations. Our results demonstrate that extracting muscle pathlets is a robust and repeatable process. Pathlets extracted from the same muscle but different subjects or from the same muscle on different days were remarkably similar and roughly appropriate for that muscle's mechanical action. Critically, muscle pathlets included extensive spatiotemporal complexity, including kinematic features before and after the present muscle activity, similar to that reported for M1 neurons. These results suggest the possibility that M1 pathlets at least partly reflect the filter-like complexities of the periphery rather than high-level representations.

Backpropagation through time and the brain.

It has long been speculated that the backpropagation-of-error algorithm (backprop) may be a model of how the brain learns. Backpropagation-through-time (BPTT) is the canonical temporal-analogue to backprop used to assign credit in recurrent neural networks in machine learning, but there's even less conviction about whether BPTT has anything to do with the brain. Even in machine learning the use of BPTT in classic neural network architectures has proven insufficient for some challenging temporal credit assignment (TCA) problems that we know the brain is capable of solving. Nonetheless, recent work in machine learning has made progress in solving difficult TCA problems by employing novel memory-based and attention-based architectures and algorithms, some of which are brain inspired. Importantly, these recent machine learning methods have been developed in the context of, and with reference to BPTT, and thus serve to strengthen BPTT's position as a useful normative guide for thinking about temporal credit assignment in artificial and biological systems alike.

Dendritic solutions to the credit assignment problem.

Guaranteeing that synaptic plasticity leads to effective learning requires a means for assigning credit to each neuron for its contribution to behavior. The 'credit assignment problem' refers to the fact that credit assignment is non-trivial in hierarchical networks with multiple stages of processing. One difficulty is that if credit signals are integrated with other inputs, then it is hard for synaptic plasticity rules to distinguish credit-related activity from non-credit-related activity. A potential solution is to use the spatial layout and non-linear properties of dendrites to distinguish credit signals from other inputs. In cortical pyramidal neurons, evidence hints that top-down feedback signals are integrated in the distal apical dendrites and have a distinct impact on spike-firing and synaptic plasticity. This suggests that the distal apical dendrites of pyramidal neurons help the brain to solve the credit assignment problem.

A deep learning framework for neuroscience.

Systems neuroscience seeks explanations for how the brain implements a wide variety of perceptual, cognitive and motor tasks. Conversely, artificial intelligence attempts to design computational systems based on the tasks they will have to solve. In artificial neural networks, the three components specified by design are the objective functions, the learning rules and the architectures. With the growing success of deep learning, which utilizes brain-inspired architectures, these three designed components have increasingly become central to how we model, engineer and optimize complex artificial learning systems. Here we argue that a greater focus on these components would also benefit systems neuroscience. We give examples of how this optimization-based framework can drive theoretical and experimental progress in neuroscience. We contend that this principled perspective on systems neuroscience will help to generate more rapid progress.

Towards deep learning with segregated dendrites.

Deep learning has led to significant advances in artificial intelligence, in part, by adopting strategies motivated by neurophysiology. However, it is unclear whether deep learning could occur in the real brain. Here, we show that a deep learning algorithm that utilizes multi-compartment neurons might help us to understand how the neocortex optimizes cost functions. Like neocortical pyramidal neurons, neurons in our model receive sensory information and higher-order feedback in electrotonically segregated compartments. Thanks to this segregation, neurons in different layers of the network can coordinate synaptic weight updates. As a result, the network learns to categorize images better than a single layer network. Furthermore, we show that our algorithm takes advantage of multilayer architectures to identify useful higher-order representations-the hallmark of deep learning. This work demonstrates that deep learning can be achieved using segregated dendritic compartments, which may help to explain the morphology of neocortical pyramidal neurons.

Random synaptic feedback weights support error backpropagation for deep learning.

The brain processes information through multiple layers of neurons. This deep architecture is representationally powerful, but complicates learning because it is difficult to identify the responsible neurons when a mistake is made. In machine learning, the backpropagation algorithm assigns blame by multiplying error signals with all the synaptic weights on each neuron's axon and further downstream. However, this involves a precise, symmetric backward connectivity pattern, which is thought to be impossible in the brain. Here we demonstrate that this strong architectural constraint is not required for effective error propagation. We present a surprisingly simple mechanism that assigns blame by multiplying errors by even random synaptic weights. This mechanism can transmit teaching signals across multiple layers of neurons and performs as effectively as backpropagation on a variety of tasks. Our results help reopen questions about how the brain could use error signals and dispel long-held assumptions about algorithmic constraints on learning.

Clonal relationships impact neuronal tuning within a phylogenetically ancient vertebrate brain structure.

Understanding how neurons acquire specific response properties is a major goal in neuroscience. Recent studies in mouse neocortex have shown that "sister neurons" derived from the same cortical progenitor cell have a greater probability of forming synaptic connections with one another and are biased to respond to similar sensory stimuli. However, it is unknown whether such lineage-based rules contribute to functional circuit organization across different species and brain regions. To address this question, we examined the influence of lineage on the response properties of neurons within the optic tectum, a visual brain area found in all vertebrates. Tectal neurons possess well-defined spatial receptive fields (RFs) whose center positions are retinotopically organized. If lineage relationships do not influence the functional properties of tectal neurons, one prediction is that the RF positions of sister neurons should be no more (or less) similar to one another than those of neighboring control neurons. To test this prediction, we developed a protocol to unambiguously identify the daughter neurons derived from single tectal progenitor cells in Xenopus laevis tadpoles. We combined this approach with in vivo two-photon calcium imaging in order to characterize the RF properties of tectal neurons. Our data reveal that the RF centers of sister neurons are significantly more similar than would be expected by chance. Ontogenetic relationships therefore influence the fine-scale topography of the retinotectal map, indicating that lineage relationships may represent a general and evolutionarily conserved principle that contributes to the organization of neural circuits.

Preference distributions of primary motor cortex neurons reflect control solutions optimized for limb biomechanics.

Neurons in monkey primary motor cortex (M1) tend to be most active for certain directions of hand movement and joint-torque loads applied to the limb. The origin and function of these biases in preference distribution are unclear but may be key to understanding the causal role of M1 in limb control. We demonstrate that these distributions arise naturally in a network model that commands muscle activity and is optimized to control movements and counter applied forces. In the model, movement and load preference distributions matching those observed empirically are only evident when particular features of the musculoskeletal system were included: limb geometry, intersegmental dynamics, and the force-length/velocity properties of muscle were dominant factors in dictating movement preferences, and the presence of biarticular muscles dictated load preferences. Our results suggest a general principle: neural activity in M1 commands muscle activity and is optimized for the physics of the motor effector.

Temporal evolution of "automatic gain-scaling".

The earliest neural response to a mechanical perturbation, the short-latency stretch response (R1: 20-45 ms), is known to exhibit "automatic gain-scaling" whereby its magnitude is proportional to preperturbation muscle activity. Because gain-scaling likely reflects an intrinsic property of the motoneuron pool (via the size-recruitment principle), counteracting this property poses a fundamental challenge for the nervous system, which must ultimately counter the absolute change in load regardless of the initial muscle activity (i.e., show no gain-scaling). Here we explore the temporal evolution of gain-scaling in a simple behavioral task where subjects stabilize their arm against different background loads and randomly occurring torque perturbations. We quantified gain-scaling in four elbow muscles (brachioradialis, biceps long, triceps lateral, triceps long) over the entire sequence of muscle activity following perturbation onset-the short-latency response, long-latency response (R2: 50-75 ms; R3: 75-105 ms), early voluntary corrections (120-180 ms), and steady-state activity (750-1250 ms). In agreement with previous observations, we found that the short-latency response demonstrated substantial gain-scaling with a threefold increase in background load resulting in an approximately twofold increase in muscle activity for the same perturbation. Following the short-latency response, we found a rapid decrease in gain-scaling starting in the long-latency epoch ( approximately 75-ms postperturbation) such that no significant gain-scaling was observed for the early voluntary corrections or steady-state activity. The rapid decrease in gain-scaling supports our recent suggestion that long-latency responses and voluntary control are inherently linked as part of an evolving sensorimotor control process through similar neural circuitry.