The application of artificial intelligence to biology and neuroscience.

Over the last decade, the artificial intelligence (AI) has undergone a revolution that is poised to transform the economy, society, and science. The pace of progress is staggering, and problems that seemed intractable just a few years ago have now been solved. The intersection between neuroscience and AI is particularly exciting.

The neuroconnectionist research programme.

Artificial neural networks (ANNs) inspired by biology are beginning to be widely used to model behavioural and neural data, an approach we call 'neuroconnectionism'. ANNs have been not only lauded as the current best models of information processing in the brain but also criticized for failing to account for basic cognitive functions. In this Perspective article, we propose that arguing about the successes and failures of a restricted set of current ANNs is the wrong approach to assess the promise of neuroconnectionism for brain science. Instead, we take inspiration from the philosophy of science, and in particular from Lakatos, who showed that the core of a scientific research programme is often not directly falsifiable but should be assessed by its capacity to generate novel insights. Following this view, we present neuroconnectionism as a general research programme centred around ANNs as a computational language for expressing falsifiable theories about brain computation. We describe the core of the programme, the underlying computational framework and its tools for testing specific neuroscientific hypotheses and deriving novel understanding. Taking a longitudinal view, we review past and present neuroconnectionist projects and their responses to challenges and argue that the research programme is highly progressive, generating new and otherwise unreachable insights into the workings of the brain.

Responses to Pattern-Violating Visual Stimuli Evolve Differently Over Days in Somata and Distal Apical Dendrites.

Scientists have long conjectured that the neocortex learns patterns in sensory data to generate top-down predictions of upcoming stimuli. In line with this conjecture, different responses to pattern-matching vs pattern-violating visual stimuli have been observed in both spiking and somatic calcium imaging data. However, it remains unknown whether these pattern-violation signals are different between the distal apical dendrites, which are heavily targeted by top-down signals, and the somata, where bottom-up information is primarily integrated. Furthermore, it is unknown how responses to pattern-violating stimuli evolve over time as an animal gains more experience with them. Here, we address these unanswered questions by analyzing responses of individual somata and dendritic branches of layer 2/3 and layer 5 pyramidal neurons tracked over multiple days in primary visual cortex of awake, behaving female and male mice. We use sequences of Gabor patches with patterns in their orientations to create pattern-matching and pattern-violating stimuli, and two-photon calcium imaging to record neuronal responses. Many neurons in both layers show large differences between their responses to pattern-matching and pattern-violating stimuli. Interestingly, these responses evolve in opposite directions in the somata and distal apical dendrites, with somata becoming less sensitive to pattern-violating stimuli and distal apical dendrites more sensitive. These differences between the somata and distal apical dendrites may be important for hierarchical computation of sensory predictions and learning, since these two compartments tend to receive bottom-up and top-down information, respectively.

Adult neurogenesis acts as a neural regularizer.

New neurons are continuously generated in the subgranular zone of the dentate gyrus throughout adulthood. These new neurons gradually integrate into hippocampal circuits, forming new naive synapses. Viewed from this perspective, these new neurons may represent a significant source of "wiring" noise in hippocampal networks. In machine learning, such noise injection is commonly used as a regularization technique. Regularization techniques help prevent overfitting training data and allow models to generalize learning to new, unseen data. Using a computational modeling approach, here we ask whether a neurogenesis-like process similarly acts as a regularizer, facilitating generalization in a category learning task. In a convolutional neural network (CNN) trained on the CIFAR-10 object recognition dataset, we modeled neurogenesis as a replacement/turnover mechanism, where weights for a randomly chosen small subset of hidden layer neurons were reinitialized to new values as the model learned to categorize 10 different classes of objects. We found that neurogenesis enhanced generalization on unseen test data compared to networks with no neurogenesis. Moreover, neurogenic networks either outperformed or performed similarly to networks with conventional noise injection (i.e., dropout, weight decay, and neural noise). These results suggest that neurogenesis can enhance generalization in hippocampal learning through noise injection, expanding on the roles that neurogenesis may have in cognition.

The study of plasticity has always been about gradients.

The experimental study of learning and plasticity has always been driven by an implicit question: how can physiological changes be adaptive and improve performance? For example, in Hebbian plasticity only synapses from presynaptic neurons that were active are changed, avoiding useless changes. Similarly, in dopamine-gated learning synapse changes depend on reward or lack thereof and do not change when everything is predictable. Within machine learning we can make the question of which changes are adaptive concrete: performance improves when changes correlate with the gradient of an objective function quantifying performance. This result is general for any system that improves through small changes. As such, physiology has always implicitly been seeking mechanisms that allow the brain to approximate gradients. Coming from this perspective we review the existing literature on plasticity-related mechanisms, and we show how these mechanisms relate to gradient estimation. We argue that gradients are a unifying idea to explain the many facets of neuronal plasticity.

Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses.

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.

Optogenetic activation of parvalbumin and somatostatin interneurons selectively restores theta-nested gamma oscillations and oscillation-induced spike timing-dependent long-term potentiation impaired by amyloid ß oligomers.

BACKGROUND: Abnormal accumulation of amyloid ß1-42 oligomers (AßO1-42), a hallmark of Alzheimer's disease, impairs hippocampal theta-nested gamma oscillations and long-term potentiation (LTP) that are believed to underlie learning and memory. Parvalbumin-positive (PV) and somatostatin-positive (SST) interneurons are critically involved in theta-nested gamma oscillogenesis and LTP induction. However, how AßO1-42 affects PV and SST interneuron circuits is unclear. Through optogenetic manipulation of PV and SST interneurons and computational modeling of the hippocampal neural circuits, we dissected the contributions of PV and SST interneuron circuit dysfunctions on AßO1-42-induced impairments of hippocampal theta-nested gamma oscillations and oscillation-induced LTP. RESULTS: Targeted whole-cell patch-clamp recordings and optogenetic manipulations of PV and SST interneurons during in vivo-like, optogenetically induced theta-nested gamma oscillations in vitro revealed that AßO1-42 causes synapse-specific dysfunction in PV and SST interneurons. AßO1-42 selectively disrupted CA1 pyramidal cells (PC)-to-PV interneuron and PV-to-PC synapses to impair theta-nested gamma oscillogenesis. In contrast, while having no effect on PC-to-SST or SST-to-PC synapses, AßO1-42 selectively disrupted SST interneuron-mediated disinhibition to CA1 PC to impair theta-nested gamma oscillation-induced spike timing-dependent LTP (tLTP). Such AßO1-42-induced impairments of gamma oscillogenesis and oscillation-induced tLTP were fully restored by optogenetic activation of PV and SST interneurons, respectively, further supporting synapse-specific dysfunctions in PV and SST interneurons. Finally, computational modeling of hippocampal neural circuits including CA1 PC, PV, and SST interneurons confirmed the experimental observations and further revealed distinct functional roles of PV and SST interneurons in theta-nested gamma oscillations and tLTP induction. CONCLUSIONS: Our results reveal that AßO1-42 causes synapse-specific dysfunctions in PV and SST interneurons and that optogenetic modulations of these interneurons present potential therapeutic targets for restoring hippocampal network oscillations and synaptic plasticity impairments in Alzheimer's disease.

Systems consolidation impairs behavioral flexibility.

Behavioral flexibility is important in a changing environment. Previous research suggests that systems consolidation, a long-term poststorage process that alters memory traces, may reduce behavioral flexibility. However, exactly how systems consolidation affects flexibility is unknown. Here, we tested how systems consolidation affects: (1) flexibility in response to value changes and (2) flexibility in response to changes in the optimal sequence of actions. Mice were trained to obtain food rewards in a Y-maze by switching nose pokes between three arms. During initial training, all arms were rewarded and mice simply had to switch arms in order to maximize rewards. Then, after either a 1 or 28 d delay, we either devalued one arm, or we reinforced a specific sequence of pokes. We found that after a 1 d delay mice adapted relatively easily to the changes. In contrast, mice given a 28 d delay struggled to adapt, especially for changes to the optimal sequence of actions. Immediate early gene imaging suggested that the 28 d mice were less reliant on their hippocampus and more reliant on their medial prefrontal cortex. These data suggest that systems consolidation reduces behavioral flexibility, particularly for changes to the optimal sequence of actions.

Irrelevance by inhibition: Learning, computation, and implications for schizophrenia.

Symptoms of schizophrenia may arise from a failure of cortical circuits to filter-out irrelevant inputs. Schizophrenia has also been linked to disruptions in cortical inhibitory interneurons, consistent with the possibility that in the normally functioning brain, these cells are in some part responsible for determining which sensory inputs are relevant versus irrelevant. Here, we develop a neural network model that demonstrates how the cortex may learn to ignore irrelevant inputs through plasticity processes affecting inhibition. The model is based on the proposal that the amount of excitatory output from a cortical circuit encodes the expected magnitude of reward or punishment ("relevance"), which can be trained using a temporal difference learning mechanism acting on feedforward inputs to inhibitory interneurons. In the model, irrelevant and blocked stimuli drive lower levels of excitatory activity compared with novel and relevant stimuli, and this difference in activity levels is lost following disruptions to inhibitory units. When excitatory units are connected to a competitive-learning output layer with a threshold, the relevance code can be shown to "gate" both learning and behavioral responses to irrelevant stimuli. Accordingly, the combined network is capable of recapitulating published experimental data linking inhibition in frontal cortex with fear learning and expression. Finally, the model demonstrates how relevance learning can take place in parallel with other types of learning, through plasticity rules involving inhibitory and excitatory components, respectively. Altogether, this work offers a theory of how the cortex learns to selectively inhibit inputs, providing insight into how relevance-assignment problems may emerge in schizophrenia.

Memory Transformation Enhances Reinforcement Learning in Dynamic Environments.

Over the course of systems consolidation, there is a switch from a reliance on detailed episodic memories to generalized schematic memories. This switch is sometimes referred to as "memory transformation." Here we demonstrate a previously unappreciated benefit of memory transformation, namely, its ability to enhance reinforcement learning in a dynamic environment. We developed a neural network that is trained to find rewards in a foraging task where reward locations are continuously changing. The network can use memories for specific locations (episodic memories) and statistical patterns of locations (schematic memories) to guide its search. We find that switching from an episodic to a schematic strategy over time leads to enhanced performance due to the tendency for the reward location to be highly correlated with itself in the short-term, but regress to a stable distribution in the long-term. We also show that the statistics of the environment determine the optimal utilization of both types of memory. Our work recasts the theoretical question of why memory transformation occurs, shifting the focus from the avoidance of memory interference toward the enhancement of reinforcement learning across multiple timescales. SIGNIFICANCE STATEMENT: As time passes, memories transform from a highly detailed state to a more gist-like state, in a process called "memory transformation." Theories of memory transformation speak to its advantages in terms of reducing memory interference, increasing memory robustness, and building models of the environment. However, the role of memory transformation from the perspective of an agent that continuously acts and receives reward in its environment is not well explored. In this work, we demonstrate a view of memory transformation that defines it as a way of optimizing behavior across multiple timescales.

Engineering protein nanoparticles for drug delivery.

Protein nanoparticles offer a highly tunable platform for engineering multifunctional drug delivery vehicles that can improve drug efficacy and reduce off-target effects. While many protein nanoparticles have demonstrated the ability to tolerate genetic and posttranslational modifications for drug delivery applications, this review will focus on three protein nanoparticles of increasing size. Each protein nanoparticle possesses distinct properties such as highly tunable stability, capacity for splitting or fusing subunits for modular surface decoration, and well-characterized conformational changes with impressive capacity for large protein cargos. While many of the genetic and posttranslational modifications leverage these protein nanoparticle's properties, the shared techniques highlight engineering approaches that have been generalized across many protein nanoparticle platforms.

Stimulus information guides the emergence of behavior-related signals in primary somatosensory cortex during learning.

Neurons in the primary cortex carry sensory- and behavior-related information, but it remains an open question how this information emerges and intersects together during learning. Current evidence points to two possible learning-related changes: sensory information increases in the primary cortex or sensory information remains stable, but its readout efficiency in association cortices increases. We investigated this question by imaging neuronal activity in mouse primary somatosensory cortex before, during, and after learning of an object localization task. We quantified sensory- and behavior-related information and estimated how much sensory information was used to instruct perceptual choices as learning progressed. We find that sensory information increases from the start of training, while choice information is mostly present in the later stages of learning. Additionally, the readout of sensory information becomes more efficient with learning as early as in the primary sensory cortex. Together, our results highlight the importance of primary cortical neurons in perceptual learning.

Excitability mediates allocation of pre-configured ensembles to a hippocampal engram supporting contextual conditioned threat in mice.

Little is understood about how engrams, sparse groups of neurons that store memories, are formed endogenously. Here, we combined calcium imaging, activity tagging, and optogenetics to examine the role of neuronal excitability and pre-existing functional connectivity on the allocation of mouse cornu ammonis area 1 (CA1) hippocampal neurons to an engram ensemble supporting a contextual threat memory. Engram neurons (high activity during recall or TRAP2-tagged during training) were more active than non-engram neurons 3 h (but not 24 h to 5 days) before training. Consistent with this, optogenetically inhibiting scFLARE2-tagged neurons active in homecage 3 h, but not 24 h, before conditioning disrupted memory retrieval, indicating that neurons with higher pre-training excitability were allocated to the engram. We also observed stable pre-configured functionally connected sub-ensembles of neurons whose activity cycled over days. Sub-ensembles that were more active before training were allocated to the engram, and their functional connectivity increased at training. Therefore, both neuronal excitability and pre-configured functional connectivity mediate allocation to an engram ensemble.

Increasing CRTC1 function in the dentate gyrus during memory formation or reactivation increases memory strength without compromising memory quality.

Memory stabilization following encoding (synaptic consolidation) or memory reactivation (reconsolidation) requires gene expression and protein synthesis (Dudai and Eisenberg, 2004; Tronson and Taylor, 2007; Nader and Einarsson, 2010; Alberini, 2011). Although consolidation and reconsolidation may be mediated by distinct molecular mechanisms (Lee et al., 2004), disrupting the function of the transcription factor CREB impairs both processes (Kida et al., 2002; Mamiya et al., 2009). Phosphorylation of CREB at Ser133 recruits CREB binding protein (CBP)/p300 coactivators to activate transcription (Chrivia et al., 1993; Parker et al., 1996). In addition to this well known mechanism, CREB regulated transcription coactivators (CRTCs), previously called transducers of regulated CREB (TORC) activity, stimulate CREB-mediated transcription, even in the absence of CREB phosphorylation. Recently, CRTC1 has been shown to undergo activity-dependent trafficking from synapses and dendrites to the nucleus in excitatory hippocampal neurons (Ch'ng et al., 2012). Despite being a powerful and specific coactivator of CREB, the role of CRTC in memory is virtually unexplored. To examine the effects of increasing CRTC levels, we used viral vectors to locally and acutely increase CRTC1 in the dorsal hippocampus dentate gyrus region of mice before training or memory reactivation in context fear conditioning. Overexpressing CRTC1 enhanced both memory consolidation and reconsolidation; CRTC1-mediated memory facilitation was context specific (did not generalize to nontrained context) and long lasting (observed after virally expressed CRTC1 dissipated). CREB overexpression produced strikingly similar effects. Therefore, increasing CRTC1 or CREB function is sufficient to enhance the strength of new, as well as established reactivated, memories without compromising memory quality.

The conjunctive trace.

Memories serve to establish some permanence to our inner lives despite the fleeting nature of subjective experience. Most neurobiological theories of memory assume that this mental permanence reflects an underlying cellular permanence. Namely, it is assumed that the cellular changes which first occur to store a memory are perpetuated for as long as the memory is stored. But is that really the case? In an opinion piece in this issue of Hippocampus, Aryeh Routtenberg raises the provocative idea that the subjective sense of memory persistence is not in fact a result of persistence at the cellular level, rather, that "supple synapses" and multiple "evanescent networks" that are forever changing are responsible for our memories. On one level, his proposal could be construed as a radical challenge to some of our most fundamental theories of the neurobiology of memory, including Donald Hebb's proposal that memories are stored by networks that strengthen their connections to increase the likelihood of the same activity patterns being recreated at a later date. However, it could also be seen as a moderating call, a call for a greater acknowledgement of the dynamic, stochastic, and distributed nature of neural networks. In this response to Routtenberg's article, we attempt to provide a clarification of the dividing line between these two interpretations of his argument, and in doing so, we provide some overview of the empirical evidence that bears on this subject. We argue that the data that exists to date favors the more moderate interpretation: that memory storage involves a process in which activity patterns are made more likely to reoccur, but that an under-appreciated reality is that mnemonic traces may continue to change and evolve over time.

Temporal encoding in deep reinforcement learning agents.

Neuroscientists have observed both cells in the brain that fire at specific points in time, known as "time cells", and cells whose activity steadily increases or decreases over time, known as "ramping cells". It is speculated that time and ramping cells support temporal computations in the brain and carry mnemonic information. However, due to the limitations in animal experiments, it is difficult to determine how these cells really contribute to behavior. Here, we show that time cells and ramping cells naturally emerge in the recurrent neural networks of deep reinforcement learning models performing simulated interval timing and working memory tasks, which have learned to estimate expected rewards in the future. We show that these cells do indeed carry information about time and items stored in working memory, but they contribute to behavior in large part by providing a dynamic representation on which policy can be computed. Moreover, the information that they do carry depends on both the task demands and the variables provided to the models. Our results suggest that time cells and ramping cells could contribute to temporal and mnemonic calculations, but the way in which they do so may be complex and unintuitive to human observers.

A Unified, Scalable Framework for Neural Population Decoding.

Our ability to use deep learning approaches to decipher neural activity would likely benefit from greater scale, in terms of both model size and datasets. However, the integration of many neural recordings into one unified model is challenging, as each recording contains the activity of different neurons from different individual animals. In this paper, we introduce a training framework and architecture designed to model the population dynamics of neural activity across diverse, large-scale neural recordings. Our method first tokenizes individual spikes within the dataset to build an efficient representation of neural events that captures the fine temporal structure of neural activity. We then employ cross-attention and a PerceiverIO backbone to further construct a latent tokenization of neural population activities. Utilizing this architecture and training framework, we construct a large-scale multi-session model trained on large datasets from seven nonhuman primates, spanning over 158 different sessions of recording from over 27,373 neural units and over 100 hours of recordings. In a number of different tasks, we demonstrate that our pretrained model can be rapidly adapted to new, unseen sessions with unspecified neuron correspondence, enabling few-shot performance with minimal labels. This work presents a powerful new approach for building deep learning tools to analyze neural data and stakes out a clear path to training at scale.

Responses of pyramidal cell somata and apical dendrites in mouse visual cortex over multiple days.

The apical dendrites of pyramidal neurons in sensory cortex receive primarily top-down signals from associative and motor regions, while cell bodies and nearby dendrites are heavily targeted by locally recurrent or bottom-up inputs from the sensory periphery. Based on these differences, a number of theories in computational neuroscience postulate a unique role for apical dendrites in learning. However, due to technical challenges in data collection, little data is available for comparing the responses of apical dendrites to cell bodies over multiple days. Here we present a dataset collected through the Allen Institute Mindscope's OpenScope program that addresses this need. This dataset comprises high-quality two-photon calcium imaging from the apical dendrites and the cell bodies of visual cortical pyramidal neurons, acquired over multiple days in awake, behaving mice that were presented with visual stimuli. Many of the cell bodies and dendrite segments were tracked over days, enabling analyses of how their responses change over time. This dataset allows neuroscientists to explore the differences between apical and somatic processing and plasticity.

Catalyzing next-generation Artificial Intelligence through NeuroAI.

Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities - inherited from over 500 million years of evolution - that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.