Area-Specificity and Plasticity of History-Dependent Value Coding During Learning.

Decision making is often driven by the subjective value of available options, a value which is formed through experience. To support this fundamental behavior, the brain must encode and maintain the subjective value. To investigate the area specificity and plasticity of value coding, we trained mice in a value-based decision task and imaged neural activity in 6 cortical areas with cellular resolution. History- and value-related signals were widespread across areas, but their strength and temporal patterns differed. In expert mice, the retrosplenial cortex (RSC) uniquely encoded history- and value-related signals with persistent population activity patterns across trials. This unique encoding of RSC emerged during task learning with a strong increase in more distant history signals. Acute inactivation of RSC selectively impaired the reward-history-based behavioral strategy. Our results indicate that RSC flexibly changes its history coding and persistently encodes value-related signals to support adaptive behaviors.

Divergent Learning-Related Transcriptional States of Cortical Glutamatergic Neurons.

Experience-dependent gene expression reshapes neural circuits, permitting the learning of knowledge and skills. Most learning involves repetitive experiences during which neurons undergo multiple stages of functional and structural plasticity. Currently, the diversity of transcriptional responses underlying dynamic plasticity during repetition-based learning is poorly understood. To close this gap, we analyzed single-nucleus transcriptomes of L2/3 glutamatergic neurons of the primary motor cortex after 3 d motor skill training or home cage control in water-restricted male mice. "Train" and "control" neurons could be discriminated with high accuracy based on expression patterns of many genes, indicating that recent experience leaves a widespread transcriptional signature across L2/3 neurons. These discriminating genes exhibited divergent modes of coregulation, differentiating neurons into discrete clusters of transcriptional states. Several states showed gene expressions associated with activity-dependent plasticity. Some of these states were also prominent in the previously published reference, suggesting that they represent both spontaneous and task-related plasticity events. Markedly, however, two states were unique to our dataset. The first state, further enriched by motor training, showed gene expression suggestive of late-stage plasticity with repeated activation, which is suitable for expected emergent neuronal ensembles that stably retain motor learning. The second state, equally found in both train and control mice, showed elevated levels of metabolic pathways and norepinephrine sensitivity, suggesting a response to common experiences specific to our experimental conditions, such as water restriction or circadian rhythm. Together, we uncovered divergent transcriptional responses across L2/3 neurons, each potentially linked with distinct features of repetition-based motor learning such as plasticity, memory, and motivation.

Learning in the Rodent Motor Cortex.

The motor cortex is far from a stable conduit for motor commands and instead undergoes significant changes during learning. An understanding of motor cortex plasticity has been advanced greatly using rodents as experimental animals. Two major focuses of this research have been on the connectivity and activity of the motor cortex. The motor cortex exhibits structural changes in response to learning, and substantial evidence has implicated the local formation and maintenance of new synapses as crucial substrates of motor learning. This synaptic reorganization translates into changes in spiking activity, which appear to result in a modification and refinement of the relationship between motor cortical activity and movement. This review presents the progress that has been made using rodents to establish the motor cortex as an adaptive structure that supports motor learning.

Super-resolution with Binary Priors: Theory and Algorithms.

The problem of super-resolution is concerned with the reconstruction of temporally/spatially localized events (or spikes) from samples of their convolution with a low-pass filter. Distinct from prior works which exploit sparsity in appropriate domains in order to solve the resulting ill-posed problem, this paper explores the role of binary priors in super-resolution, where the spike (or source) amplitudes are assumed to be binary-valued. Our study is inspired by the problem of neural spike deconvolution, but also applies to other applications such as symbol detection in hybrid millimeter wave communication systems. This paper makes several theoretical and algorithmic contributions to enable binary super-resolution with very few measurements. Our results show that binary constraints offer much stronger identifiability guarantees than sparsity, allowing us to operate in "extreme compression" regimes, where the number of measurements can be significantly smaller than the sparsity level of the spikes. To ensure exact recovery in this "extreme compression" regime, it becomes necessary to design algorithms that exactly enforce binary constraints without relaxation. In order to overcome the ensuing computational challenges, we consider a first order auto-regressive filter (which appears in neural spike deconvolution), and exploit its special structure. This results in a novel formulation of the super-resolution binary spike recovery in terms of binary search in one dimension. We perform numerical experiments that validate our theory and also show the benefits of binary constraints in neural spike deconvolution from real calcium imaging datasets.

Characterizing Cortex-Wide Dynamics with Wide-Field Calcium Imaging.

The brain functions through coordinated activity among distributed regions. Wide-field calcium imaging, combined with improved genetically encoded calcium indicators, allows sufficient signal-to-noise ratio and spatiotemporal resolution to afford a unique opportunity to capture cortex-wide dynamics on a moment-by-moment basis in behaving animals. Recent applications of this approach have been uncovering cortical dynamics at unprecedented scales during various cognitive processes, ranging from relatively simple sensorimotor integration to more complex decision-making tasks. In this review, we will highlight recent scientific advances enabled by wide-field calcium imaging in behaving mice. We then summarize several technical considerations and future opportunities for wide-field imaging to uncover large-scale circuit dynamics.

Disengagement of Motor Cortex during Long-Term Learning Tracks the Performance Level of Learned Movements.

Not all movements require the motor cortex for execution. Intriguingly, dependence on motor cortex of a given movement is not fixed, but instead can dynamically change over the course of long-term learning. For instance, rodent forelimb movements that initially require motor cortex can become independent of the motor cortex after an extended period of training. However, it remains unclear whether long-term neural changes rendering the motor cortex dispensable are a simple function of the training length. To address this issue, we trained mice (both male and female) to perform two distinct forelimb movements, forward versus downward reaches with a joystick, concomitantly over several weeks, and then compared the involvement of the motor cortex between the two movements. Most mice achieved different levels of motor performance between the two movements after long-term training. Of the two movements, the one that achieved higher trial-to-trial consistency (i.e., consistent-direction movement) was significantly less affected by inactivation of motor cortex than the other (i.e., variable-direction movement). Two-photon calcium imaging of motor cortical neurons revealed that the consistent-direction movement activates fewer neurons, producing weaker and less consistent population activity than the variable-direction movement. Together, the motor cortex was less engaged and less necessary for learned movements that achieved higher levels of consistency. Thus, the long-term reorganization of neural circuits that frees the motor cortex from the learned movement is not a mere function of training length. Rather, this reorganization tracks the level of motor performance that the animal achieves during training.SIGNIFICANCE STATEMENT Long-term training of a movement reshapes motor circuits, disengaging motor cortex potentially for automatized execution of the learned movement. Acquiring new motor skills often involves learning of multiple movements (e.g., forehand and backhand strokes when learning tennis), but different movements do not always improve at the same time nor reach the same level of proficiency. Here we showed that the involvement of motor cortex after long-term training differs between similar yet distinct movements that reached different levels of expertise. Motor cortex was less engaged and less necessary for the more proficient movement. Thus, disengagement of motor cortex is not a simple function of training time, but instead tracks the level of expertise of a learned movement.

Stability of motor cortex network states during learning-associated neural reorganizations.

A substantial reorganization of neural activity and neuron-to-movement relationship in motor cortical circuits accompanies the emergence of reproducible movement patterns during motor learning. Little is known about how this tempest of neural activity restructuring impacts the stability of network states in recurrent cortical circuits. To investigate this issue, we reanalyzed data in which we recorded for 14 days via population calcium imaging the activity of the same neural populations of pyramidal neurons in layer 2/3 and layer 5 of forelimb motor and premotor cortex in mice during the daily learning of a lever-press task. We found that motor cortex network states remained stable with respect to the critical network state during the extensive reorganization of both neural population activity and its relation to lever movement throughout learning. Specifically, layer 2/3 cortical circuits unceasingly displayed robust evidence for operating at the critical network state, a regime that maximizes information capacity and transmission and provides a balance between network robustness and flexibility. In contrast, layer 5 circuits operated away from the critical network state for all 14 days of recording and learning. In conclusion, this result indicates that the wide-ranging malleability of synapses, neurons, and neural connectivity during learning operates within the constraint of a stable and layer-specific network state regarding dynamic criticality, and suggests that different cortical layers operate under distinct constraints because of their specialized goals.NEW & NOTEWORTHY The neural activity reorganizes throughout motor learning, but how this reorganization impacts the stability of network states is unclear. We used two-photon calcium imaging to investigate how the network states in layer 2/3 and layer 5 of forelimb motor and premotor cortex are modulated by motor learning. We show that motor cortex network states are layer-specific and constant regarding criticality during neural activity reorganization, and suggests that layer-specific constraints could be motivated by different functions.

Emergence of reproducible spatiotemporal activity during motor learning.

The motor cortex is capable of reliably driving complex movements yet exhibits considerable plasticity during motor learning. These observations suggest that the fundamental relationship between motor cortex activity and movement may not be fixed but is instead shaped by learning; however, to what extent and how motor learning shapes this relationship are not fully understood. Here we addressed this issue by using in vivo two-photon calcium imaging to monitor the activity of the same population of hundreds of layer 2/3 neurons while mice learned a forelimb lever-press task over two weeks. Excitatory and inhibitory neurons were identified by transgenic labelling. Inhibitory neuron activity was relatively stable and balanced local excitatory neuron activity on a movement-by-movement basis, whereas excitatory neuron activity showed higher dynamism during the initial phase of learning. The dynamics of excitatory neurons during the initial phase involved the expansion of the movement-related population which explored various activity patterns even during similar movements. This was followed by a refinement into a smaller population exhibiting reproducible spatiotemporal sequences of activity. This pattern of activity associated with the learned movement was unique to expert animals and not observed during similar movements made during the naive phase, and the relationship between neuronal activity and individual movements became more consistent with learning. These changes in population activity coincided with a transient increase in dendritic spine turnover in these neurons. Our results indicate that a novel and reproducible activity-movement relationship develops as a result of motor learning, and we speculate that synaptic plasticity within the motor cortex underlies the emergence of reproducible spatiotemporal activity patterns for learned movements. These results underscore the profound influence of learning on the way that the cortex produces movements.

Ultra-low Impedance Graphene Microelectrodes with High Optical Transparency for Simultaneous Deep 2-photon Imaging in Transgenic Mice.

The last decades have witnessed substantial progress in optical technologies revolutionizing our ability to record and manipulate neural activity in genetically modified animal models. Meanwhile, human studies mostly rely on electrophysiological recordings of cortical potentials, which cannot be inferred from optical recordings, leading to a gap between our understanding of dynamics of microscale populations and brain-scale neural activity. By enabling concurrent integration of electrical and optical modalities, transparent graphene microelectrodes can close this gap. However, the high impedance of graphene constitutes a big challenge towards the widespread use of this technology. Here, we experimentally demonstrate that this high impedance of graphene microelectrodes is fundamentally limited by quantum capacitance. We overcome this quantum capacitance limit by creating a parallel conduction path using platinum nanoparticles. We achieve a 100 times reduction in graphene electrode impedance, while maintaining the high optical transparency crucial for deep 2-photon microscopy. Using a transgenic mouse model, we demonstrate simultaneous electrical recording of cortical activity with high fidelity while imaging calcium signals at various cortical depths right beneath the transparent microelectrodes. Multimodal analysis of Ca2+ spikes and cortical surface potentials offers unique opportunities to bridge our understanding of cellular dynamics and brain-scale neural activity.

Neuromodulation in Chemosensory Pathways.

Interactions with the environment depend not only on sensory perception of external stimuli but also on processes of neuromodulation regulated by the internal state of an organism. These processes allow regulation of stimulus detection to match the demands of an organism influenced by its general brain state (satiety, wakefulness/sleep state, attentiveness, arousal, learning etc.). The sense of smell is initiated by sensory neurons located in the nasal cavity that recognize environmental odorants and project axons into the olfactory bulb (OB), where they form synapses with several types of neurons. Modulations of early synaptic circuits are particularly important since these can affect all subsequent processing steps. While the precise mechanisms have not been fully elucidated, work from many labs has demonstrated that the activity of neurons in the OB and cortex can be modulated by different factors inducing specific changes to olfactory information processing. The symposium "Neuromodulation in Chemosensory Pathways" at the International Symposium on Olfaction and Taste (ISOT 2016) highlighted some of the most recent advances in state-dependent network modulations of the mouse olfactory system including modulation mediated by specific neurotransmitters and neuroendocrine molecules, involving pharmacological, electrophysiological, learning, and behavioral approaches.

Learning-related fine-scale specificity imaged in motor cortex circuits of behaving mice.

Cortical neurons form specific circuits, but the functional structure of this microarchitecture and its relation to behaviour are poorly understood. Two-photon calcium imaging can monitor activity of spatially defined neuronal ensembles in the mammalian cortex. Here we applied this technique to the motor cortex of mice performing a choice behaviour. Head-fixed mice were trained to lick in response to one of two odours, and to withhold licking for the other odour. Mice routinely showed significant learning within the first behavioural session and across sessions. Microstimulation and trans-synaptic tracing identified two non-overlapping candidate tongue motor cortical areas. Inactivating either area impaired voluntary licking. Imaging in layer 2/3 showed neurons with diverse response types in both areas. Activity in approximately half of the imaged neurons distinguished trial types associated with different actions. Many neurons showed modulation coinciding with or preceding the action, consistent with their involvement in motor control. Neurons with different response types were spatially intermingled. Nearby neurons (within approximately 150 mum) showed pronounced coincident activity. These temporal correlations increased with learning within and across behavioural sessions, specifically for neuron pairs with similar response types. We propose that correlated activity in specific ensembles of functionally related neurons is a signature of learning-related circuit plasticity. Our findings reveal a fine-scale and dynamic organization of the frontal cortex that probably underlies flexible behaviour.

Local and global predictors of synapse elimination during motor learning.

During learning, synaptic connections between excitatory neurons in the brain display considerable dynamism, with new connections being added and old connections eliminated. Synapse elimination offers an opportunity to understand the features of synapses that the brain deems dispensable. However, with limited observations of synaptic activity and plasticity in vivo, the features of synapses subjected to elimination remain poorly understood. Here, we examined the functional basis of synapse elimination in the apical dendrites of L2/3 neurons in the primary motor cortex throughout motor learning. We found no evidence that synapse elimination is facilitated by a lack of activity or other local forms of plasticity. Instead, eliminated synapses display asynchronous activity with nearby synapses, suggesting that functional synaptic clustering is a critical component of synapse survival. In addition, eliminated synapses show delayed activity timing with respect to postsynaptic output. Thus, synaptic inputs that fail to be co-active with their neighboring synapses or are mistimed with neuronal output are targeted for elimination.

High-density transparent graphene arrays for predicting cellular calcium activity at depth from surface potential recordings.

Optically transparent neural microelectrodes have facilitated simultaneous electrophysiological recordings from the brain surface with the optical imaging and stimulation of neural activity. A remaining challenge is to scale down the electrode dimensions to the single-cell size and increase the density to record neural activity with high spatial resolution across large areas to capture nonlinear neural dynamics. Here we developed transparent graphene microelectrodes with ultrasmall openings and a large, transparent recording area without any gold extensions in the field of view with high-density microelectrode arrays up to 256 channels. We used platinum nanoparticles to overcome the quantum capacitance limit of graphene and to scale down the microelectrode diameter to 20 µm. An interlayer-doped double-layer graphene was introduced to prevent open-circuit failures. We conducted multimodal experiments, combining the recordings of cortical potentials of microelectrode arrays with two-photon calcium imaging of the mouse visual cortex. Our results revealed that visually evoked responses are spatially localized for high-frequency bands, particularly for the multiunit activity band. The multiunit activity power was found to be correlated with cellular calcium activity. Leveraging this, we employed dimensionality reduction techniques and neural networks to demonstrate that single-cell and average calcium activities can be decoded from surface potentials recorded by high-density transparent graphene arrays.

Exponential history integration with diverse temporal scales in retrosplenial cortex supports hyperbolic behavior.

Animals use past experience to guide future choices. The integration of experiences typically follows a hyperbolic, rather than exponential, decay pattern with a heavy tail for distant history. Hyperbolic integration affords sensitivity to both recent environmental dynamics and long-term trends. However, it is unknown how the brain implements hyperbolic integration. We found that mouse behavior in a foraging task showed hyperbolic decay of past experience, but the activity of cortical neurons showed exponential decay. We resolved this apparent mismatch by observing that cortical neurons encode history information with heterogeneous exponential time constants that vary across neurons. A model combining these diverse timescales recreated the heavy-tailed, hyperbolic history integration observed in behavior. In particular, the time constants of retrosplenial cortex (RSC) neurons best matched the behavior, and optogenetic inactivation of RSC uniquely reduced behavioral history dependence. These results indicate that behavior-relevant history information is maintained across multiple timescales in parallel and that RSC is a critical reservoir of information guiding decision-making.

Meta-reinforcement learning via orbitofrontal cortex.

The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca2+/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.

AlphaTracker: a multi-animal tracking and behavioral analysis tool.

Computer vision has emerged as a powerful tool to elevate behavioral research. This protocol describes a computer vision machine learning pipeline called AlphaTracker, which has minimal hardware requirements and produces reliable tracking of multiple unmarked animals, as well as behavioral clustering. AlphaTracker pairs a top-down pose-estimation software combined with unsupervised clustering to facilitate behavioral motif discovery that will accelerate behavioral research. All steps of the protocol are provided as open-source software with graphic user interfaces or implementable with command-line prompts. Users with a graphical processing unit (GPU) can model and analyze animal behaviors of interest in less than a day. AlphaTracker greatly facilitates the analysis of the mechanism of individual/social behavior and group dynamics.

PatchWarp: Corrections of non-uniform image distortions in two-photon calcium imaging data by patchwork affine transformations.

Complex distortions on calcium imaging often impair image registration accuracy. Here, we developed a registration algorithm, PatchWarp, to robustly correct slow image distortion for calcium imaging data. PatchWarp is a two-step algorithm with rigid and non-rigid image registrations. To correct non-uniform image distortions, it splits the imaging field and estimates the best affine transformation matrix for each of the subfields. The distortion-corrected subfields are stitched together like a patchwork to reconstruct the distortion-corrected imaging field. We show that PatchWarp robustly corrects image distortions of calcium imaging data collected from various cortical areas through glass window or gradient-index (GRIN) lens with a higher accuracy than existing non-rigid algorithms. Furthermore, it provides a fully automated method of registering images from different imaging sessions for longitudinal neural activity analyses. PatchWarp improves the quality of neural activity analyses and is useful as a general approach to correct image distortions in a wide range of disciplines.

Longitudinal two-photon calcium imaging with ultra-large cranial window for head-fixed mice.

Neural activity is heterogeneous across different cortical areas and can change during learning. Here, we describe a protocol for longitudinal in vivo two-photon calcium imaging with an ultra-large cranial window that exposes most of the dorsal cortex in head-fixed mice. The large cranial window allows optical access to any dorsal cortical areas in individual mice. This protocol enables longitudinal tracking of neural activity from various cortical areas at cellular resolution to understand the cortical computations during behavioral tasks. For complete details on the use and execution of this protocol, please refer to Hattori et al. (2019), and Hattori and Komiyama, 2022a.

Context-dependent persistency as a coding mechanism for robust and widely distributed value coding.

Task-related information is widely distributed across the brain with different coding properties, such as persistency. We found in mice that coding persistency of action history and value was variable across areas, learning phases, and task context, with the highest persistency in the retrosplenial cortex of expert mice performing value-based decisions where history needs to be maintained across trials. Persistent coding also emerged in artificial networks trained to perform mouse-like reinforcement learning. Persistency allows temporally untangled value representations in neuronal manifolds where population activity exhibits cyclic trajectories that transition along the value axis after action outcomes, collectively forming cylindrical dynamics. Simulations indicated that untangled persistency facilitates robust value retrieval by downstream networks. Even leakage of persistently maintained value through non-specific connectivity could contribute to the brain-wide distributed value coding with different levels of persistency. These results reveal that context-dependent, untangled persistency facilitates reliable signal coding and its distribution across the brain.

Global and subtype-specific modulation of cortical inhibitory neurons regulated by acetylcholine during motor learning.

Inhibitory neurons (INs) consist of distinct subtypes with unique functions. Previous studies on INs mainly focused on single brain regions, and thus it remains unclear whether the modulation of IN subtypes occurs globally across multiple regions. Here, we monitored the activity of different cortical IN subtypes at both macroscale and microscale in mice learning a lever-press task. Learning evoked a global modulation of IN subtypes throughout the cortex. The initial learning phase involved strong activation of vasoactive intestinal peptide-expressing INs (VIP-INs) and weak activation of somatostatin-expressing INs (SOM-INs). Inactivating VIP-INs increased SOM-IN activity and impaired initial learning. Concurrently, cortical cholinergic inputs from the basal forebrain were initially more active but became less engaged over learning. Manipulation of the cholinergic system impaired motor learning and differentially altered activity of IN subtypes. These results reveal that motor learning involves a global and subtype-specific modulation on cortical INs regulated by the cholinergic system.