The neural substrates of how model-based learning affects risk taking: Functional coupling between right cerebellum and left caudate.

Higher executive control capacity allows people to appropriately evaluate risk and avoid both excessive risk aversion and excessive risk-taking. The neural mechanisms underlying this relationship between executive function and risk taking are still unknown. We used voxel-based morphometry (VBM) analysis combined with resting-state functional connectivity (rs-FC) to evaluate how one component of executive function, model-based learning, relates to risk taking. We measured individuals' use of the model-based learning system with the two-step task, and risk taking with the Balloon Analogue Risk Task. Behavioral results indicated that risk taking was positively correlated with the model-based weighting parameter ω. The VBM results showed a positive association between model-based learning and gray matter volume in the right cerebellum (RCere) and left inferior parietal lobule (LIPL). Functional connectivity results suggested that the coupling between RCere and the left caudate (LCAU) was correlated with both model-based learning and risk taking. Mediation analysis indicated that RCere-LCAU functional connectivity completely mediated the effect of model-based learning on risk taking. These results indicate that learners who favor model-based strategies also engage in more appropriate risky behaviors through interactions between reward-based learning, error-based learning and executive control subserved by a caudate, cerebellar and parietal network.

Right Lateral Cerebellum Represents Linguistic Predictability.

Mounting evidence indicates that posterolateral portions of the cerebellum (right Crus I/II) contribute to language processing, but the nature of this role remains unclear. Based on a well-supported theory of cerebellar motor function, which ascribes to the cerebellum a role in short-term prediction through internal modeling, we hypothesize that right cerebellar Crus I/II supports prediction of upcoming sentence content. We tested this hypothesis using event-related fMRI in male and female human subjects by manipulating the predictability of written sentences. Our design controlled for motor planning and execution, as well as for linguistic features and working memory load; it also allowed separation of the prediction interval from the presentation of the final sentence item. In addition, three further fMRI tasks captured semantic, phonological, and orthographic processing to shed light on the nature of the information processed. As hypothesized, activity in right posterolateral cerebellum correlated with the predictability of the upcoming target word. This cerebellar region also responded to prediction error during the outcome of the trial. Further, this region was engaged in phonological, but not semantic or orthographic, processing. This is the first imaging study to demonstrate a right cerebellar contribution in language comprehension independently from motor, cognitive, and linguistic confounds. These results complement our work using other methodologies showing cerebellar engagement in linguistic prediction and suggest that internal modeling of phonological representations aids language production and comprehension.SIGNIFICANCE STATEMENT The cerebellum is traditionally seen as a motor structure that allows for smooth movement by predicting upcoming signals. However, the cerebellum is also consistently implicated in nonmotor functions such as language and working memory. Using fMRI, we identify a cerebellar area that is active when words are predicted and when these predictions are violated. This area is active in a separate task that requires phonological processing, but not in tasks that require semantic or visuospatial processing. Our results support the idea of prediction as a unifying cerebellar function in motor and nonmotor domains. We provide new insights by linking the cerebellar role in prediction to its role in verbal working memory, suggesting that these predictions involve phonological processing.

Vicarious reinforcement learning signals when instructing others.

Reinforcement learning (RL) theory posits that learning is driven by discrepancies between the predicted and actual outcomes of actions (prediction errors [PEs]). In social environments, learning is often guided by similar RL mechanisms. For example, teachers monitor the actions of students and provide feedback to them. This feedback evokes PEs in students that guide their learning. We report the first study that investigates the neural mechanisms that underpin RL signals in the brain of a teacher. Neurons in the anterior cingulate cortex (ACC) signal PEs when learning from the outcomes of one's own actions but also signal information when outcomes are received by others. Does a teacher's ACC signal PEs when monitoring a student's learning? Using fMRI, we studied brain activity in human subjects (teachers) as they taught a confederate (student) action-outcome associations by providing positive or negative feedback. We examined activity time-locked to the students' responses, when teachers infer student predictions and know actual outcomes. We fitted a RL-based computational model to the behavior of the student to characterize their learning, and examined whether a teacher's ACC signals when a student's predictions are wrong. In line with our hypothesis, activity in the teacher's ACC covaried with the PE values in the model. Additionally, activity in the teacher's insula and ventromedial prefrontal cortex covaried with the predicted value according to the student. Our findings highlight that the ACC signals PEs vicariously for others' erroneous predictions, when monitoring and instructing their learning. These results suggest that RL mechanisms, processed vicariously, may underpin and facilitate teaching behaviors.

Multimodal connectivity of motor learning-related dorsal premotor cortex.

The dorsal premotor cortex (dPMC) is a key region for motor learning and sensorimotor integration, yet we have limited understanding of its functional interactions with other regions. Previous work has started to examine functional connectivity in several brain areas using resting state functional connectivity (RSFC) and meta-analytical connectivity modelling (MACM). More recently, structural covariance (SC) has been proposed as a technique that may also allow delineation of functional connectivity. Here, we applied these three approaches to provide a comprehensive characterization of functional connectivity with a seed in the left dPMC that a previous meta-analysis of functional neuroimaging studies has identified as playing a key role in motor learning. Using data from two sources (the Rockland sample, containing resting state data and anatomical scans from 132 participants, and the BrainMap database, which contains peak activation foci from over 10,000 experiments), we conducted independent whole-brain functional connectivity mapping analyses of a dPMC seed. RSFC and MACM revealed similar connectivity maps spanning prefrontal, premotor, and parietal regions, while the SC map identified more widespread frontal regions. Analyses indicated a relatively consistent pattern of functional connectivity between RSFC and MACM that was distinct from that identified by SC. Notably, results indicate that the seed is functionally connected to areas involved in visuomotor control and executive functions, suggesting that the dPMC acts as an interface between motor control and cognition.

Computational Investigations of Learning and Synchronization in Cognitive Control.

Complex cognition requires binding together of stimulus, action, and other features, across different time scales. Several implementations of such binding have been proposed in the literature, most prominently synaptic binding (learning) and synchronization. Biologically plausible accounts of how these different types of binding interact in the human brain are still lacking. To this end, we adopt a computational approach to investigate the impact of learning and synchronization on both behavioral (reaction time, error rate) and neural (θ power) measures. We train four models varying in their ability to learn and synchronize for an extended period of time on three seminal action control paradigms varying in difficulty. Learning, but not synchronization, proved essential for behavioral improvement. Synchronization however boosts performance of difficult tasks, avoiding the computational pitfalls of catastrophic interference. At the neural level, θ power decreases with practice but increases with task difficulty. Our simulation results bring new insights in how different types of binding interact in different types of tasks, and how this is translated in both behavioral and neural metrics.

Neural Signatures of Cognitive Flexibility and Reward Sensitivity Following Nicotinic Receptor Stimulation in Dependent Smokers: A Randomized Trial.

Importance: Withdrawal from nicotine is an important contributor to smoking relapse. Understanding how reward-based decision making is affected by abstinence and by pharmacotherapies such as nicotine replacement therapy and varenicline tartrate may aid cessation treatment. Objective: To independently assess the effects of nicotine dependence and stimulation of the nicotinic acetylcholine receptor on the ability to interpret valence information (reward sensitivity) and subsequently alter behavior as reward contingencies change (cognitive flexibility) in a probabilistic reversal learning task. Design, Setting, and Participants: Nicotine-dependent smokers and nonsmokers completed a probabilistic reversal learning task during acquisition of functional magnetic resonance imaging (fMRI) in a 2-drug, double-blind placebo-controlled crossover design conducted from January 21, 2009, to September 29, 2011. Smokers were abstinent from cigarette smoking for 12 hours for all sessions. In a fully Latin square fashion, participants in both groups underwent MRI twice while receiving varenicline and twice while receiving a placebo pill, wearing either a nicotine or a placebo patch. Imaging analysis was performed from June 15, 2015, to August 10, 2016. Main Outcome and Measures: A well-established computational model captured effects of smoking status and administration of nicotine and varenicline on probabilistic reversal learning choice behavior. Neural effects of smoking status, nicotine, and varenicline were tested for on MRI contrasts that captured reward sensitivity and cognitive flexibility. Results: The study included 24 nicotine-dependent smokers (12 women and 12 men; mean [SD] age, 35.8 [9.9] years) and 20 nonsmokers (10 women and 10 men; mean [SD] age, 30.4 [7.2] years). Computational modeling indicated that abstinent smokers were biased toward response shifting and that their decisions were less sensitive to the available evidence, suggesting increased impulsivity during withdrawal. These behavioral impairments were mitigated with nicotine and varenicline. Similarly, decreased mesocorticolimbic activity associated with cognitive flexibility in abstinent smokers was restored to the level of nonsmokers following stimulation of nicotinic acetylcholine receptors (familywise error-corrected P < .05). Conversely, neural signatures of decreased reward sensitivity in smokers (vs nonsmokers; familywise error-corrected P < .05) in the dorsal striatum and anterior cingulate cortex were not mitigated by nicotine or varenicline. Conclusions and Relevance: There was a double dissociation between the effects of chronic nicotine dependence on neural representations of reward sensitivity and acute effects of stimulation of nicotinic acetylcholine receptors on behavioral and neural signatures of cognitive flexibility in smokers. These chronic and acute pharmacologic effects were observed in overlapping mesocorticolimbic regions, suggesting that available pharmacotherapies may alleviate deficits in the same circuitry for certain mental computations but not for others. Trial Registration: clinicaltrials.gov Identifier: NCT00830739.

Cerebellar BOLD signal during the acquisition of a new lexicon predicts its early consolidation.

Cerebellar contributions to language are presently poorly understood, but it has been argued that the cerebellar role in motor learning can be extended to learning in cognitive and linguistic domains. Here, we used fMRI to investigate whether the cerebellum is recruited in mapping novel words onto existing semantic concepts. On separate days, participants performed a Basque vocabulary learning task and a control English synonym task in the MRI scanner. Learning-related BOLD activity was found in left inferior frontal gyrus, bilateral insula, pre-SMA, left superior parietal cortex, right caudate, the right cerebellar vermis and right cerebellar Crus II. The extent to which the cerebellar regions, but not the cerebral areas, were recruited during learning correlated positively with participants' off-line improvement in performance after the learning task. These data provide evidence for a cerebellar role in lexical learning, and suggest that the right cerebellum may contribute toward consolidation of lexico-semantic associations in the language network.

Cerebellar transcranial magnetic stimulation: the role of coil geometry and tissue depth.

BACKGROUND: While transcranial magnetic stimulation (TMS) coil geometry has important effects on the evoked magnetic field, no study has systematically examined how different coil designs affect the effectiveness of cerebellar stimulation. HYPOTHESIS: The depth of the cerebellar targets will limit efficiency. Angled coils designed to stimulate deeper tissue are more effective in eliciting cerebellar stimulation. METHODS: Experiment 1 examined basic input-output properties of the figure-of-eight, batwing and double-cone coils, assessed with stimulation of motor cortex. Experiment 2 assessed the ability of each coil to activate cerebellum, using cerebellar-brain inhibition (CBI). Experiment 3 mapped distances from the scalp to cerebellar and motor cortical targets in a sample of 100 subjects' structural magnetic resonance images. RESULTS: Experiment 1 showed batwing and double-cone coils have significantly lower resting motor thresholds, and recruitment curves with steeper slopes than the figure-of-eight coil. Experiment 2 showed the double-cone coil was the most efficient for eliciting CBI. The batwing coil induced CBI only at higher stimulus intensities. The figure-of-eight coil did not elicit reliable CBI. Experiment 3 confirmed that cerebellar tissue is significantly deeper than primary motor cortex tissue, and we provide a map of scalp-to-target distances. CONCLUSIONS: The double-cone and batwing coils designed to stimulate deeper tissue can effectively stimulate cerebellar targets. The double-cone coil was found to be most effective. The depth map provides a guide to the accessible regions of the cerebellar volume. These results can guide coil selection and stimulation parameters when designing cerebellar TMS studies.