Medial prefrontal cortex suppresses reward-seeking behavior with risk of punishment by reducing sensitivity to reward.

Reward-seeking behavior is frequently associated with risk of punishment. There are two types of punishment: positive punishment, which is defined as addition of an aversive stimulus, and negative punishment, involves the omission of a rewarding outcome. Although the medial prefrontal cortex (mPFC) is important in avoiding punishment, whether it is important for avoiding both positive and negative punishment and how it contributes to such avoidance are not clear. In this study, we trained male mice to perform decision-making tasks under the risks of positive (air-puff stimulus) and negative (reward omission) punishment, and modeled their behavior with reinforcement learning. Following the training, we pharmacologically inhibited the mPFC. We found that pharmacological inactivation of mPFC enhanced the reward-seeking choice under the risk of positive, but not negative, punishment. In reinforcement learning models, this behavioral change was well-explained as an increase in sensitivity to reward, rather than a decrease in the strength of aversion to punishment. Our results suggest that mPFC suppresses reward-seeking behavior by reducing sensitivity to reward under the risk of positive punishment.

Change detection in the primate auditory cortex through feedback of prediction error signals.

Although cortical feedback signals are essential for modulating feedforward processing, no feedback error signal across hierarchical cortical areas has been reported. Here, we observed such a signal in the auditory cortex of awake common marmoset during an oddball paradigm to induce auditory duration mismatch negativity. Prediction errors to a deviant tone presentation were generated as offset calcium responses of layer 2/3 neurons in the rostral parabelt (RPB) of higher-order auditory cortex, while responses to non-deviant tones were strongly suppressed. Within several hundred milliseconds, the error signals propagated broadly into layer 1 of the primary auditory cortex (A1) and accumulated locally on top of incoming auditory signals. Blockade of RPB activity prevented deviance detection in A1. Optogenetic activation of RPB following tone presentation nonlinearly enhanced A1 tone response. Thus, the feedback error signal is critical for automatic detection of unpredicted stimuli in physiological auditory processing and may serve as backpropagation-like learning.

Layer 5 Intratelencephalic Neurons in the Motor Cortex Stably Encode Skilled Movement.

The primary motor cortex (M1) and the dorsal striatum play a critical role in motor learning and the retention of learned behaviors. Motor representations of corticostriatal ensembles emerge during motor learning. In the coordinated reorganization of M1 and the dorsal striatum for motor learning, layer 5a (L5a) which connects M1 to the ipsilateral and contralateral dorsal striatum, should be a key layer. Although M1 L5a neurons represent movement-related activity in the late stage of learning, it is unclear whether the activity is retained as a memory engram. Here, using Tlx3-Cre male transgenic mice, we conducted two-photon calcium imaging of striatum-projecting L5a intratelencephalic (IT) neurons in forelimb M1 during late sessions of a self-initiated lever-pull task and in sessions after 6 d of nontraining following the late sessions. We found that trained male animals exhibited stable motor performance before and after the nontraining days. At the same time, we found that M1 L5a IT neurons strongly represented the well-learned forelimb movement but not uninstructed orofacial movements. A subset of M1 L5a IT neurons consistently coded the well-learned forelimb movement before and after the nontraining days. Inactivation of M1 IT neurons after learning impaired task performance when the lever was made heavier or when the target range of the pull distance was narrowed. These results suggest that a subset of M1 L5a IT neurons continuously represent skilled movement after learning and serve to fine-tune the kinematics of well-learned movement.SIGNIFICANCE STATEMENT Motor memory persists even when it is not used for a while. IT neurons in L5a of the M1 gradually come to represent skilled forelimb movements during motor learning. However, it remains to be determined whether these changes persist over a long period and how these neurons contribute to skilled movements. Here, we show that a subset of M1 L5a IT neurons retain information for skilled forelimb movements even after nontraining days. Furthermore, suppressing the activity of these neurons during skilled forelimb movements impaired behavioral stability and adaptability. Our results suggest the importance of M1 L5a IT neurons for tuning skilled forelimb movements over a long period.

Dynamic organization of cerebellar climbing fiber response and synchrony in multiple functional components reduces dimensions for reinforcement learning.

Cerebellar climbing fibers convey diverse signals, but how they are organized in the compartmental structure of the cerebellar cortex during learning remains largely unclear. We analyzed a large amount of coordinate-localized two-photon imaging data from cerebellar Crus II in mice undergoing 'Go/No-go' reinforcement learning. Tensor component analysis revealed that a majority of climbing fiber inputs to Purkinje cells were reduced to only four functional components, corresponding to accurate timing control of motor initiation related to a Go cue, cognitive error-based learning, reward processing, and inhibition of erroneous behaviors after a No-go cue. Changes in neural activities during learning of the first two components were correlated with corresponding changes in timing control and error learning across animals, indirectly suggesting causal relationships. Spatial distribution of these components coincided well with boundaries of Aldolase-C/zebrin II expression in Purkinje cells, whereas several components are mixed in single neurons. Synchronization within individual components was bidirectionally regulated according to specific task contexts and learning stages. These findings suggest that, in close collaborations with other brain regions including the inferior olive nucleus, the cerebellum, based on anatomical compartments, reduces dimensions of the learning space by dynamically organizing multiple functional components, a feature that may inspire new-generation AI designs.

Transition of distinct context-dependent ensembles from secondary to primary motor cortex in skilled motor performance.

When voluntary movements are executed under different contexts, different context-dependent signals are thought to weaken from secondary motor cortex (M2) to primary motor cortex (M1). However, it is unclear how the different contexts are processed from M2 to M1 to execute skilled movement. We conduct two-photon calcium imaging of M2 and M1 in mice performing internally generated and external-cue-triggered movements. Context dependency is consistently high in M2 L2/3 neurons and consistently low in M1 pyramidal tract neurons. By contrast, context dependency in M2 → M1 axons and M1 L2/3 neurons increases as task performance improves. In addition, the context dependency of M1 L2/3, but not M2 → M1 axons, is associated with fine-movement proficiency. The increase in context dependency correlates with stabilization of the context-dependent population activity and an increase in the neurons that strongly encode contextual and motor information. Thus, emergence of distinct context-dependent ensembles may be necessary for the context-to-motor transformation that facilitates skilled motor performance.

Silent microscopy to explore a brain that hears butterflies' wings.

A silent two-photon laser-scanning microscopy system, which eliminates mechanical vibrations in the audible range, has enabled the detection of auditory cortical neurons with responses at sound pressure levels as low as 5 dB in nonhuman primates.

Neuronal representations of reward-predicting cues and outcome history with movement in the frontal cortex.

Transformation of sensory inputs to goal-directed actions requires estimation of sensory-cue values based on outcome history. We conduct wide-field and two-photon calcium imaging of the mouse neocortex during classical conditioning with two cues with different water-reward probabilities. Although licking movement dominates the area-averaged activity over the whole dorsal neocortex, the dorsomedial frontal cortex (dmFrC) affects other dorsal frontal cortical activities, and its inhibition extinguishes differences in anticipatory licking between the cues. Many dorsal frontal and medial prefrontal cortical neurons are task related. Subsets of these neurons are more excited by the low-reward-predicting cue or unrewarded outcomes than by the high-reward-predicting cue or rewarded outcomes, respectively. Task-related activities of these neurons and the others are counterbalanced, so that population activity appears dominated by licking. The reward-predicting cue and outcome history are most strongly represented in dmFrC. Our results suggest that dmFrC is crucial for initiating cortical processes to select or inhibit action.

Tb3+-doped fluorescent glass for biology.

Optical investigation and manipulation constitute the core of biological experiments. Here, we introduce a new borosilicate glass material that contains the rare-earth ion terbium(III) (Tb3+), which emits green fluorescence upon blue light excitation, similar to green fluorescent protein (GFP), and thus is widely compatible with conventional biological research environments. Micropipettes made of Tb3+-doped glass allowed us to target GFP-labeled cells for single-cell electroporation, single-cell transcriptome analysis (Patch-seq), and patch-clamp recording under real-time fluorescence microscopic control. The glass also exhibited potent third harmonic generation upon infrared laser excitation and was usable for online optical targeting of fluorescently labeled neurons in the in vivo neocortex. Thus, Tb3+-doped glass simplifies many procedures in biological experiments.

Non-action Learning: Saving Action-Associated Cost Serves as a Covert Reward.

"To do or not to do" is a fundamental decision that has to be made in daily life. Behaviors related to multiple "to do" choice tasks have long been explained by reinforcement learning, and "to do or not to do" tasks such as the go/no-go task have also been recently discussed within the framework of reinforcement learning. In this learning framework, alternative actions and/or the non-action to take are determined by evaluating explicitly given (overt) reward and punishment. However, we assume that there are real life cases in which an action/non-action is repeated, even though there is no obvious reward or punishment, because implicitly given outcomes such as saving physical energy and regret (we refer to this as "covert reward") can affect the decision-making. In the current task, mice chose to pull a lever or not according to two tone cues assigned with different water reward probabilities (70% and 30% in condition 1, and 30% and 10% in condition 2). As the mice learned, the probability that they would choose to pull the lever decreased (<0.25) in trials with a 30% reward probability cue (30% cue) in condition 1, and in trials with a 10% cue in condition 2, but increased (>0.8) in trials with a 70% cue in condition 1 and a 30% cue in condition 2, even though a non-pull was followed by neither an overt reward nor avoidance of overt punishment in any trial. This behavioral tendency was not well explained by a combination of commonly used Q-learning models, which take only the action choice with an overt reward outcome into account. Instead, we found that the non-action preference of the mice was best explained by Q-learning models, which regarded the non-action as the other choice, and updated non-action values with a covert reward. We propose that "doing nothing" can be actively chosen as an alternative to "doing something," and that a covert reward could serve as a reinforcer of "doing nothing."

Structural dynamics and stability of corticocortical and thalamocortical axon terminals during motor learning.

Synaptic plasticity is the cellular basis of learning and memory. When animals learn a novel motor skill, synaptic modifications are induced in the primary motor cortex (M1), and new postsynaptic dendritic spines relevant to motor memory are formed in the early stage of learning. However, it is poorly understood how presynaptic axonal boutons are formed, eliminated, and maintained during motor learning, and whether long-range corticocortical and thalamocortical axonal boutons show distinct structural changes during learning. In this study, we conducted two-photon imaging of presynaptic boutons of long-range axons in layer 1 (L1) of the mouse M1 during the 7-day learning of an accelerating rotarod task. The training-period-averaged rate of formation of boutons on axons projecting from the secondary motor cortical area increased, while the average rate of elimination of those from the motor thalamus (thalamic boutons) decreased. In particular, the elimination rate of thalamic boutons during days 4-7 was lower than that in untrained mice, and the fraction of pre-existing thalamic boutons that survived until day 7 was higher than that in untrained mice. Our results suggest that the late stabilization of thalamic boutons in M1 contributes to motor skill learning.

Common marmoset as a model primate for study of the motor control system.

The common marmoset (Callithrix jacchus), a New World monkey, is emerging as a promising animal model for biomedical and neuroscience research. This species shares its basic brain architecture, including the organization of the motor cortical areas and the connections between these and other areas, with humans and other primates. Its small and lissencephalic cerebral cortex is suitable for the application of modern biological techniques. Optogenetic stimulation of the motor cortex induces forelimb movements, and two-photon calcium imaging allows detection of forelimb movement-related activity in multiple motor cortical neurons. The common marmoset also has a large repertoire of forelimb-related behaviors and vocal communications. Thus, the common marmoset is a good model for research into voluntary forelimb movements, social behaviors, and their dysfunctions.

Motor learning requires myelination to reduce asynchrony and spontaneity in neural activity.

Myelination increases the conduction velocity in long-range axons and is prerequisite for many brain functions. Impaired myelin regulation or impairment of myelin itself is frequently associated with deficits in learning and cognition in neurological and psychiatric disorders. However, it has not been revealed what perturbation of neural activity induced by myelin impairment causes learning deficits. Here, we measured neural activity in the motor cortex during motor learning in transgenic mice with a subtle impairment of their myelin. This deficit in myelin impaired motor learning, and was accompanied by a decrease in the amplitude of movement-related activity and an increase in the frequency of spontaneous activity. Thalamocortical axons showed variability in axonal conduction with a large spread in the timing of postsynaptic cortical responses. Repetitive pairing of forelimb movements with optogenetic stimulation of thalamocortical axon terminals restored motor learning. Thus, myelin regulation helps to maintain the synchrony of cortical spike-time arrivals through long-range axons, facilitating the propagation of the information required for learning. Our results revealed the pathological neuronal circuit activity with impaired myelin and suggest the possibility that pairing of noninvasive brain stimulation with relevant behaviors may ameliorate cognitive and behavioral abnormalities in diseases with impaired myelination.

Arm movements induced by noninvasive optogenetic stimulation of the motor cortex in the common marmoset.

Optogenetics is now a fundamental tool for investigating the relationship between neuronal activity and behavior. However, its application to the investigation of motor control systems in nonhuman primates is rather limited, because optogenetic stimulation of cortical neurons in nonhuman primates has failed to induce or modulate any hand/arm movements. Here, we used a tetracycline-inducible gene expression system carrying CaMKII promoter and the gene encoding a Channelrhodopsin-2 variant with fast kinetics in the common marmoset, a small New World monkey. In an awake state, forelimb movements could be induced when Channelrhodopsin-2-expressing neurons in the motor cortex were illuminated by blue laser light with a spot diameter of 1 mm or 2 mm through a cranial window without cortical invasion. Forelimb muscles responded 10 ms to 50 ms after photostimulation onset. Long-duration (500 ms) photostimulation induced discrete forelimb movements that could be markerlessly tracked with charge-coupled device cameras and a deep learning algorithm. Long-duration photostimulation mapping revealed that the primary motor cortex is divided into multiple domains that can induce hand and elbow movements in different directions. During performance of a forelimb movement task, movement trajectories were modulated by weak photostimulation, which did not induce visible forelimb movements at rest, around the onset of task-relevant movement. The modulation was biased toward the movement direction induced by the strong photostimulation. Combined with calcium imaging, all-optical interrogation of motor circuits should be possible in behaving marmosets.

Modular organization of cerebellar climbing fiber inputs during goal-directed behavior.

The cerebellum has a parasagittal modular architecture characterized by precisely organized climbing fiber (CF) projections that are congruent with alternating aldolase C/zebrin II expression. However, the behavioral relevance of CF inputs into individual modules remains poorly understood. Here, we used two-photon calcium imaging in the cerebellar hemisphere Crus II in mice performing an auditory go/no-go task to investigate the functional differences in CF inputs to modules. CF signals in medial modules show anticipatory decreases, early increases, secondary increases, and reward-related increases or decreases, which represent quick motor initiation, go cues, fast motor behavior, and positive reward outcomes. CF signals in lateral modules show early increases and reward-related decreases, which represent no-go and/or go cues and positive reward outcomes. The boundaries of CF functions broadly correspond to those of aldolase C patterning. These results indicate that spatially segregated CF inputs in different modules play distinct roles in the execution of goal-directed behavior.

Super-wide-field two-photon imaging with a micro-optical device moving in post-objective space.

Wide-field imaging of neural activity at a cellular resolution is a current challenge in neuroscience. To address this issue, wide-field two-photon microscopy has been developed; however, the field size is limited by the objective size. Here, we develop a micro-opto-mechanical device that rotates within the post-objective space between the objective and brain tissue. Two-photon microscopy with this device enables sub-second sequential calcium imaging of left and right mouse sensory forelimb areas 6 mm apart. When imaging the rostral and caudal motor forelimb areas (RFA and CFA) 2 mm apart, we found high pairwise correlations in spontaneous activity between RFA and CFA neurons and between an RFA neuron and its putative axons in CFA. While mice performed a sound-triggered forelimb-movement task, the population activity between RFA and CFA covaried across trials, although the field-averaged activity was similar across trials. The micro-opto-mechanical device in the post-objective space provides a novel and flexible design to clarify the correlation structure between distant brain areas at subcellular and population levels.

Thalamocortical Axonal Activity in Motor Cortex Exhibits Layer-Specific Dynamics during Motor Learning.

The thalamus is the hub through which neural signals are transmitted from the basal ganglia and cerebellum to the neocortex. However, thalamocortical axonal activity during motor learning remains largely undescribed. We conducted two-photon calcium imaging of thalamocortical axonal activity in the motor cortex of mice learning a self-initiated lever-pull task. Layer 1 (L1) axons came to exhibit activity at lever-pull initiation and termination, while layer 3 (L3) axons did so at lever-pull initiation. L1 population activity had a sequence structure related to both lever-pull duration and reproducibility. Stimulation of the substantia nigra pars reticulata activated more L1 than L3 axons, whereas deep cerebellar nuclei (DCN) stimulation did the opposite. Lesions to either the dorsal striatum or the DCN impaired motor learning and disrupted temporal dynamics in both layers. Thus, layer-specific thalamocortical signals evolve with the progression of learning, which requires both the basal ganglia and cerebellar activities.

Two-photon imaging of neuronal activity in motor cortex of marmosets during upper-limb movement tasks.

Two-photon imaging in behaving animals has revealed neuronal activities related to behavioral and cognitive function at single-cell resolution. However, marmosets have posed a challenge due to limited success in training on motor tasks. Here we report the development of protocols to train head-fixed common marmosets to perform upper-limb movement tasks and simultaneously perform two-photon imaging. After 2-5 months of training sessions, head-fixed marmosets can control a manipulandum to move a cursor to a target on a screen. We conduct two-photon calcium imaging of layer 2/3 neurons in the motor cortex during this motor task performance, and detect task-relevant activity from multiple neurons at cellular and subcellular resolutions. In a two-target reaching task, some neurons show direction-selective activity over the training days. In a short-term force-field adaptation task, some neurons change their activity when the force field is on. Two-photon calcium imaging in behaving marmosets may become a fundamental technique for determining the spatial organization of the cortical dynamics underlying action and cognition.

In vivo wide-field calcium imaging of mouse thalamocortical synapses with an 8 K ultra-high-definition camera.

In vivo wide-field imaging of neural activity with a high spatio-temporal resolution is a challenge in modern neuroscience. Although two-photon imaging is very powerful, high-speed imaging of the activity of individual synapses is mostly limited to a field of approximately 200 µm on a side. Wide-field one-photon epifluorescence imaging can reveal neuronal activity over a field of ≥1 mm2 at a high speed, but is not able to resolve a single synapse. Here, to achieve a high spatio-temporal resolution, we combine an 8 K ultra-high-definition camera with spinning-disk one-photon confocal microscopy. This combination allowed us to image a 1 mm2 field with a pixel resolution of 0.21 µm at 60 fps. When we imaged motor cortical layer 1 in a behaving head-restrained mouse, calcium transients were detected in presynaptic boutons of thalamocortical axons sparsely labeled with GCaMP6s, although their density was lower than when two-photon imaging was used. The effects of out-of-focus fluorescence changes on calcium transients in individual boutons appeared minimal. Axonal boutons with highly correlated activity were detected over the 1 mm2 field, and were probably distributed on multiple axonal arbors originating from the same thalamic neuron. This new microscopy with an 8 K ultra-high-definition camera should serve to clarify the activity and plasticity of widely distributed cortical synapses.

Silencing of FUS in the common marmoset (Callithrix jacchus) brain via stereotaxic injection of an adeno-associated virus encoding shRNA.

Fused in sarcoma (FUS) is an RNA binding protein that is involved in frontotemporal lobar degeneration (FTLD) and amyotrophic lateral sclerosis (ALS). To establish the common marmoset (Callithrix jacchus) as a model for FTLD, we generated a stereotaxic injection-based marmoset model of FUS-silencing. We designed shRNAs against the marmoset FUS gene and generated an AAV9 virus encoding the most effective shRNA against FUS (shFUS). The AAV encoding shFUS (AAV-shFUS) was introduced into the frontal cortex of young adult marmosets, whereas AAV encoding a control shRNA was injected into the contralateral side. We obtained approximately 70-80% silencing of FUS following AAV-shFUS injection. Interestingly, FUS-silencing provoked a proliferation of astrocytes and microglias. Since FTLD is characterized by various emotional deficits, it would be helpful to establish a marmoset model of FUS-silencing in various brain tissues for investigating the pathomechanism of higher cognitive and behavioral dysfunction.