Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts.

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Menstrual cycle-driven hormone concentrations co-fluctuate with white and gray matter architecture changes across the whole brain.

Cyclic fluctuations in hypothalamic-pituitary-gonadal axis (HPG-axis) hormones exert powerful behavioral, structural, and functional effects through actions on the mammalian central nervous system. Yet, very little is known about how these fluctuations alter the structural nodes and information highways of the human brain. In a study of 30 naturally cycling women, we employed multidimensional diffusion and T1-weighted imaging during three estimated menstrual cycle phases (menses, ovulation, and mid-luteal) to investigate whether HPG-axis hormone concentrations co-fluctuate with alterations in white matter (WM) microstructure, cortical thickness (CT), and brain volume. Across the whole brain, 17ß-estradiol and luteinizing hormone (LH) concentrations were directly proportional to diffusion anisotropy (µFA; 17ß-estradiol: ß1 = 0.145, highest density interval (HDI) = [0.211, 0.4]; LH: ß1 = 0.111, HDI = [0.157, 0.364]), while follicle-stimulating hormone (FSH) was directly proportional to CT (ß1 = 0 .162, HDI = [0.115, 0.678]). Within several individual regions, FSH and progesterone demonstrated opposing relationships with mean diffusivity (Diso) and CT. These regions mainly reside within the temporal and occipital lobes, with functional implications for the limbic and visual systems. Finally, progesterone was associated with increased tissue (ß1 = 0.66, HDI = [0.607, 15.845]) and decreased cerebrospinal fluid (CSF; ß1 = -0.749, HDI = [-11.604, -0.903]) volumes, with total brain volume remaining unchanged. These results are the first to report simultaneous brain-wide changes in human WM microstructure and CT coinciding with menstrual cycle-driven hormone rhythms. Effects were observed in both classically known HPG-axis receptor-dense regions (medial temporal lobe, prefrontal cortex) and in other regions located across frontal, occipital, temporal, and parietal lobes. Our results suggest that HPG-axis hormone fluctuations may have significant structural impacts across the entire brain.

Diffusion MRI head motion correction methods are highly accurate but impacted by denoising and sampling scheme.

Head motion correction is particularly challenging in diffusion-weighted MRI (dMRI) scans due to the dramatic changes in image contrast at different gradient strengths and directions. Head motion correction is typically performed using a Gaussian Process model implemented in FSL's Eddy. Recently, the 3dSHORE-based SHORELine method was introduced that does not require shell-based acquisitions, but it has not been previously benchmarked. Here we perform a comprehensive evaluation of both methods on realistic simulations of a software fiber phantom that provides known ground-truth head motion. We demonstrate that both methods perform remarkably well, but that performance can be impacted by sampling scheme and the extent of head motion and the denoising strategy applied before head motion correction. Furthermore, we find Eddy benefits from denoising the data first with MP-PCA. In sum, we provide the most extensive known benchmarking of dMRI head motion correction, together with extensive simulation data and a reproducible workflow. PRACTITIONER POINTS: Both Eddy and SHORELine head motion correction methods performed quite well on a large variety of simulated data. Denoising with MP-PCA can improve head motion correction performance when Eddy is used. SHORELine effectively corrects motion in non-shelled diffusion spectrum imaging data.

A practical evaluation of measures derived from compressed sensing diffusion spectrum imaging.

Diffusion Spectrum Imaging (DSI) using dense Cartesian sampling of q-space has been shown to provide important advantages for modeling complex white matter architecture. However, its adoption has been limited by the lengthy acquisition time required. Sparser sampling of q-space combined with compressed sensing (CS) reconstruction techniques has been proposed as a way to reduce the scan time of DSI acquisitions. However prior studies have mainly evaluated CS-DSI in post-mortem or non-human data. At present, the capacity for CS-DSI to provide accurate and reliable measures of white matter anatomy and microstructure in the living human brain remains unclear. We evaluated the accuracy and inter-scan reliability of 6 different CS-DSI schemes that provided up to 80% reductions in scan time compared to a full DSI scheme. We capitalized on a dataset of 26 participants who were scanned over eight independent sessions using a full DSI scheme. From this full DSI scheme, we subsampled images to create a range of CS-DSI images. This allowed us to compare the accuracy and inter-scan reliability of derived measures of white matter structure (bundle segmentation, voxel-wise scalar maps) produced by the CS-DSI and the full DSI schemes. We found that CS-DSI estimates of both bundle segmentations and voxel-wise scalars were nearly as accurate and reliable as those generated by the full DSI scheme. Moreover, we found that the accuracy and reliability of CS-DSI was higher in white matter bundles that were more reliably segmented by the full DSI scheme. As a final step, we replicated the accuracy of CS-DSI in a prospectively acquired dataset (n = 20, scanned once). Together, these results illustrate the utility of CS-DSI for reliably delineating in vivo white matter architecture in a fraction of the scan time, underscoring its promise for both clinical and research applications.

QSIPrep: an integrative platform for preprocessing and reconstructing diffusion MRI data.

Diffusion-weighted magnetic resonance imaging (dMRI) is the primary method for noninvasively studying the organization of white matter in the human brain. Here we introduce QSIPrep, an integrative software platform for the processing of diffusion images that is compatible with nearly all dMRI sampling schemes. Drawing on a diverse set of software suites to capitalize on their complementary strengths, QSIPrep facilitates the implementation of best practices for processing of diffusion images.

Interhemispheric Structural Connectivity Underlies Motor Recovery after Stroke.

OBJECTIVE: Although ample evidence highlights that the ipsilesional corticospinal tract (CST) plays a crucial role in motor recovery after stroke, studies on cortico-cortical motor connections remain scarce and provide inconclusive results. Given their unique potential to serve as structural reserve enabling motor network reorganization, the question arises whether cortico-cortical connections may facilitate motor control depending on CST damage. METHODS: Diffusion spectrum imaging (DSI) and a novel compartment-wise analysis approach were used to quantify structural connectivity between bilateral cortical core motor regions in chronic stroke patients. Basal and complex motor control were differentially assessed. RESULTS: Both basal and complex motor performance were correlated with structural connectivity between bilateral premotor areas and ipsilesional primary motor cortex (M1) as well as interhemispheric M1 to M1 connectivity. Whereas complex motor skills depended on CST integrity, a strong association between M1 to M1 connectivity and basal motor control was observed independent of CST integrity especially in patients who underwent substantial motor recovery. Harnessing the informational wealth of cortico-cortical connectivity facilitated the explanation of both basal and complex motor control. INTERPRETATION: We demonstrate for the first time that distinct aspects of cortical structural reserve enable basal and complex motor control after stroke. In particular, recovery of basal motor control may be supported via an alternative route through contralesional M1 and non-crossing fibers of the contralesional CST. Our findings help to explain previous conflicting interpretations regarding the functional role of the contralesional M1 and highlight the potential of cortico-cortical structural connectivity as a future biomarker for motor recovery post-stroke. ANN NEUROL 2023;94:785-797.

Combining Repetition Suppression and Pattern Analysis Provides New Insights into the Role of M1 and Parietal Areas in Skilled Sequential Actions.

How does the brain change during learning? In functional magnetic resonance imaging (fMRI) studies, both multivariate pattern analysis (MVPA) and repetition suppression (RS) have been used to detect changes in neuronal representations. In the context of motor sequence learning, the two techniques have provided discrepant findings: pattern analysis showed that only premotor and parietal regions, but not primary motor cortex (M1), develop a representation of trained sequences. In contrast, RS suggested trained sequence representations in all these regions. Here, we applied both analysis techniques to a five-week finger sequence training study, in which participants executed each sequence twice before switching to a different sequence. Both RS and pattern analysis indicated learning-related changes for parietal areas, but only RS showed a difference between trained and untrained sequences in M1. A more fine-grained analysis, however, revealed that the RS effect in M1 reflects a fundamentally different process than in parietal areas. On the first execution, M1 represents especially the first finger of each sequence, likely reflecting preparatory processes. This effect dramatically reduces during the second execution. In contrast, parietal areas represent the identity of a sequence, and this representation stays relatively stable on the second execution. These results suggest that the RS effect does not reflect a trained sequence representation in M1, but rather a preparatory signal for movement initiation. More generally, our study demonstrates that across regions RS can reflect different representational changes in the neuronal population code, emphasizing the importance of combining pattern analysis and RS techniques.SIGNIFICANCE STATEMENT Previous studies using pattern analysis have suggested that primary motor cortex (M1) does not represent learnt sequential actions. However, a study using repetition suppression (RS) has reported M1 changes during motor sequence learning. Combining both techniques, we first replicate the discrepancy between them, with learning-related changes in M1 in RS, but not pattern dissimilarities. We further analyzed the representational changes with repetition, and found that the RS effects differ across regions. M1's activity represents the starting finger of the sequence, an effect that vanishes with repetition. In contrast, activity patterns in parietal areas exhibit sequence dependency, which persists with repetition. These results demonstrate the importance of combining RS and pattern analysis to understand the function of brain regions.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T.

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Representational Neural Mapping of Dexterous Grasping Before Lifting in Humans.

The ability of humans to reach and grasp objects in their environment has been the mainstay paradigm for characterizing the neural circuitry driving object-centric actions. Although much is known about hand shaping, a persistent question is how the brain orchestrates and integrates the grasp with lift forces of the fingers in a coordinated manner. The objective of the current study was to investigate how the brain represents grasp configuration and lift force during a dexterous object-centric action in a large sample of male and female human subjects. BOLD activity was measured as subjects used a precision-grasp to lift an object with a center of mass (CoM) on the left or right with the goal of minimizing tilting the object. The extent to which grasp configuration and lift force varied between left and right CoM conditions was manipulated by grasping the object collinearly (requiring a non-collinear force distribution) or non-collinearly (requiring more symmetrical forces). Bayesian variational representational similarity analyses on fMRI data assessed the evidence that a set of cortical and cerebellar regions were sensitive to grasp configuration or lift force differences between CoM conditions at differing time points during a grasp to lift action. In doing so, we reveal strong evidence that grasping and lift force are not represented by spatially separate functionally specialized regions, but by the same regions at differing time points. The coordinated grasp to lift effort is shown to be under dorsolateral (PMv and AIP) more than dorsomedial control, and under SPL7, somatosensory PSC, ventral LOC and cerebellar control.SIGNIFICANCE STATEMENT Clumsy disasters such as spilling, dropping, and crushing during our daily interactions with objects are a rarity rather than the norm. These disasters are avoided in part as a result of our orchestrated anticipatory efforts to integrate and coordinate grasping and lifting of object interactions, all before the lift of an object even commences. How the brain orchestrates this integration process has been largely neglected by historical approaches independently and solely focusing on reaching and grasping and the neural principles that guide them. Here, we test the extent to which grasping and lifting are represented in a spatially or temporally distinct manner and identified strong evidence for the consecutive emergence of sensitivity to grasping, then lifting within the same region.

Progesterone shapes medial temporal lobe volume across the human menstrual cycle.

The rhythmic production of sex steroid hormones is a central feature of the mammalian endocrine system. In rodents and nonhuman primates, sex hormones are powerful regulators of hippocampal subfield morphology. However, it remains unknown whether intrinsic fluctuations in sex hormones alter hippocampal morphology in the human brain. In a series of dense-sampling studies, we used high-resolution imaging of the medial temporal lobe (MTL) to determine whether endogenous fluctuations (Study 1) and exogenous manipulation (Study 2) of sex hormones alter MTL volume over time. Across the menstrual cycle, intrinsic fluctuations in progesterone were associated with volumetric changes in CA2/3, entorhinal, perirhinal, and parahippocampal cortex. Chronic progesterone suppression abolished these cycle-dependent effects and led to pronounced volumetric changes in entorhinal cortex and CA2/3 relative to freely cycling conditions. No associations with estradiol were observed. These results establish progesterone's ability to rapidly and dynamically shape MTL morphology across the human menstrual cycle.

Functional reorganization of brain networks across the human menstrual cycle.

The brain is an endocrine organ, sensitive to the rhythmic changes in sex hormone production that occurs in most mammalian species. In rodents and nonhuman primates, estrogen and progesterone's impact on the brain is evident across a range of spatiotemporal scales. Yet, the influence of sex hormones on the functional architecture of the human brain is largely unknown. In this dense-sampling, deep phenotyping study, we examine the extent to which endogenous fluctuations in sex hormones alter intrinsic brain networks at rest in a woman who underwent brain imaging and venipuncture for 30 consecutive days. Standardized regression analyses illustrate estrogen and progesterone's widespread associations with functional connectivity. Time-lagged analyses examined the temporal directionality of these relationships and suggest that cortical network dynamics (particularly in the Default Mode and Dorsal Attention Networks, whose hubs are densely populated with estrogen receptors) are preceded-and perhaps driven-by hormonal fluctuations. A similar pattern of associations was observed in a follow-up study one year later. Together, these results reveal the rhythmic nature in which brain networks reorganize across the human menstrual cycle. Neuroimaging studies that densely sample the individual connectome have begun to transform our understanding of the brain's functional organization. As these results indicate, taking endocrine factors into account is critical for fully understanding the intrinsic dynamics of the human brain.

Sympathetic involvement in time-constrained sequential foraging.

Appraising sequential offers relative to an unknown future opportunity and a time cost requires an optimization policy that draws on a learned estimate of an environment's richness. Converging evidence points to a learning asymmetry, whereby estimates of this richness update with a bias toward integrating positive information. We replicate this bias in a sequential foraging (prey selection) task and probe associated activation within the sympathetic branch of the autonomic system, using trial-by-trial measures of simultaneously recorded cardiac autonomic physiology. We reveal a unique adaptive role for the sympathetic branch in learning. It was specifically associated with adaptation to a deteriorating environment: it correlated with both the rate of negative information integration in belief estimates and downward changes in moment-to-moment environmental richness, and was predictive of optimal performance on the task. The findings are consistent with a framework whereby autonomic function supports the learning demands of prey selection.

Harm to self outweighs benefit to others in moral decision making.

How we make decisions that have direct consequences for ourselves and others forms the moral foundation of our society. Whereas economic theory contends that humans aim at maximizing their own gains, recent seminal psychological work suggests that our behavior is instead hyperaltruistic: We are more willing to sacrifice gains to spare others from harm than to spare ourselves from harm. To investigate how such egoistic and hyperaltruistic tendencies influence moral decision making, we investigated trade-off decisions combining monetary rewards and painful electric shocks, administered to the participants themselves or an anonymous other. Whereas we replicated the notion of hyperaltruism (i.e., the willingness to forego reward to spare others from harm), we observed strongly egoistic tendencies in participants' unwillingness to harm themselves for others' benefit. The moral principle guiding intersubject trade-off decision making observed in our study is best described as egoistically biased altruism, with important implications for our understanding of economic and social interactions in our society.

Neural Representations of Sensorimotor Memory- and Digit Position-Based Load Force Adjustments Before the Onset of Dexterous Object Manipulation.

Anticipatory load forces for dexterous object manipulation in humans are modulated based on visual object property cues, sensorimotor memories of previous experiences with the object, and, when digit positioning varies from trial to trial, the integrating of this sensed variability with force modulation. Studies of the neural representations encoding these anticipatory mechanisms have not considered these mechanisms separately from each other or from feedback mechanisms emerging after lift onset. Here, representational similarity analyses of fMRI data were used to identify neural representations of sensorimotor memories and the sensing and integration of digit position. Cortical activity and movement kinematics were measured as 20 human subjects (11 women) minimized tilt of a symmetrically shaped object with a concealed asymmetric center of mass (CoM, left and right sided). This task required generating compensatory torques in opposite directions, which, without helpful visual CoM cues, relied primarily on sensorimotor memories of the same object and CoM. Digit position was constrained or unconstrained, the latter of which required modulating forces beyond what can be recalled from sensorimotor memories to compensate for digit position variability. Ventral premotor (PMv), somatosensory, and cerebellar lobule regions (CrusII, VIIIa) were sensitive to anticipatory behaviors that reflect sensorimotor memory content, as shown by larger voxel pattern differences for unmatched than matched CoM conditions. Cerebellar lobule I-IV, Broca area 44, and PMv showed greater voxel pattern differences for unconstrained than constrained grasping, which suggests their sensitivity to monitor the online coincidence of planned and actual digit positions and correct for a mismatch by force modulation.SIGNIFICANCE STATEMENT To pick up a water glass without slipping, tipping, or spilling requires anticipatory planning of fingertip load forces before the lift commences. This anticipation relies on object visual properties (e.g., mass/mass distribution), sensorimotor memories built from previous experiences (especially when object properties cannot be inferred visually), and online sensing of where the digits are positioned. There is limited understanding of how the brain represents each of these anticipatory mechanisms. We used fMRI measures of regional brain patterns and digit position kinematics before lift onset of an object with nonsalient visual cues specifically to isolate sensorimotor memories and integration of sensed digit position with force modulation. In doing so, we localized neural representations encoding these anticipatory mechanisms for dexterous object manipulation.

Spatial coherence of oriented white matter microstructure: Applications to white matter regions associated with genetic similarity.

We present a method to discover differences between populations with respect to the spatial coherence of their oriented white matter microstructure in arbitrarily shaped white matter regions. This method is applied to diffusion MRI scans of a subset of the Human Connectome Project dataset: 57 pairs of monozygotic and 52 pairs of dizygotic twins. After controlling for morphological similarity between twins, we identify 3.7% of all white matter as being associated with genetic similarity (35.1 k voxels, p<10-4, false discovery rate 1.5%), 75% of which spatially clusters into twenty-two contiguous white matter regions. Furthermore, we show that the orientation similarity within these regions generalizes to a subset of 47 pairs of non-twin siblings, and show that these siblings are on average as similar as dizygotic twins. The regions are located in deep white matter including the superior longitudinal fasciculus, the optic radiations, the middle cerebellar peduncle, the corticospinal tract, and within the anterior temporal lobe, as well as the cerebellum, brain stem, and amygdalae. These results extend previous work using undirected fractional anisotrophy for measuring putative heritable influences in white matter. Our multidirectional extension better accounts for crossing fiber connections within voxels. This bottom up approach has at its basis a novel measurement of coherence within neighboring voxel dyads between subjects, and avoids some of the fundamental ambiguities encountered with tractographic approaches to white matter analysis that estimate global connectivity.

Beyond modularity: Fine-scale mechanisms and rules for brain network reconfiguration.

The human brain is in constant flux, as distinct areas engage in transient communication to support basic behaviors as well as complex cognition. The collection of interactions between cortical and subcortical areas forms a functional brain network whose topology evolves with time. Despite the nontrivial dynamics that are germane to this networked system, experimental evidence demonstrates that functional interactions organize into putative brain systems that facilitate different facets of cognitive computation. We hypothesize that such dynamic functional networks are organized around a set of rules that constrain their spatial architecture - which brain regions may functionally interact - and their temporal architecture - how these interactions fluctuate over time. To objectively uncover these organizing principles, we apply an unsupervised machine learning approach called non-negative matrix factorization to time-evolving, resting state functional networks in 20 healthy subjects. This machine learning approach automatically groups temporally co-varying functional interactions into subgraphs that represent putative topological modes of dynamic functional architecture. We find that subgraphs are stratified based on both the underlying modular organization and the topographical distance of their strongest interactions: while many subgraphs are largely contained within modules, others span between modules and are expressed differently over time. The relationship between dynamic subgraphs and modular architecture is further highlighted by the ability of time-varying subgraph expression to explain inter-individual differences in module reorganization. Collectively, these results point to the critical role that subgraphs play in constraining the topography and topology of functional brain networks. More broadly, this machine learning approach opens a new door for understanding the architecture of dynamic functional networks during both task and rest states, and for probing alterations of that architecture in disease.

Analytic tractography: A closed-form solution for estimating local white matter connectivity with diffusion MRI.

White matter structures composed of myelinated axons in the living human brain are primarily studied by diffusion-weighted MRI (dMRI). These long-range projections are typically characterized in a two-step process: dMRI signal is used to estimate the orientation of axon segments within each voxel, then these local orientations are linked together to estimate the spatial extent of putative white matter bundles. Tractography, the process of tracing bundles across voxels, either requires computationally expensive (probabilistic) simulations to model uncertainty in fiber orientation or ignores it completely (deterministic). Furthermore, simulation necessarily generates a finite number of trajectories, introducing "simulation error" to trajectory estimates. Here we introduce a method to analytically (via a closed-form solution) take an orientation distribution function (ODF) from each voxel and calculate the probabilities that a trajectory projects from a voxel into each directly adjacent voxels. We validate our method by demonstrating experimentally that probabilistic simulations converge to our analytically computed transition probabilities at the voxel level as the number of simulated seeds increases. We then show that our method accurately calculates the ground-truth transition probabilities from a publicly available phantom dataset. As a demonstration, we incorporate our analytic method for voxel transition probabilities into the Voxel Graph framework, creating a quantitative framework for assessing white matter structure, which we call "analytic tractography". The long-range connectivity problem is reduced to finding paths in a graph whose adjacency structure reflects voxel-to-voxel analytic transition probabilities. We demonstrate that this approach performs comparably to the current most widely-used probabilistic and deterministic approaches at a fraction of the computational cost. We also demonstrate that analytic tractography works on multiple diffusion sampling schemes, reconstruction method or parameters used to define paths. Open source software compatible with popular dMRI reconstruction software is provided.

Predicting future learning from baseline network architecture.

Human behavior and cognition result from a complex pattern of interactions between brain regions. The flexible reconfiguration of these patterns enables behavioral adaptation, such as the acquisition of a new motor skill. Yet, the degree to which these reconfigurations depend on the brain's baseline sensorimotor integration is far from understood. Here, we asked whether spontaneous fluctuations in sensorimotor networks at baseline were predictive of individual differences in future learning. We analyzed functional MRI data from 19 participants prior to six weeks of training on a new motor skill. We found that visual-motor connectivity was inversely related to learning rate: sensorimotor autonomy at baseline corresponded to faster learning in the future. Using three additional scans, we found that visual-motor connectivity at baseline is a relatively stable individual trait. These results suggest that individual differences in motor skill learning can be predicted from sensorimotor autonomy at baseline prior to task execution.

Brain state flexibility accompanies motor-skill acquisition.

Learning requires the traversal of inherently distinct cognitive states to produce behavioral adaptation. Yet, tools to explicitly measure these states with non-invasive imaging - and to assess their dynamics during learning - remain limited. Here, we describe an approach based on a distinct application of graph theory in which points in time are represented by network nodes, and similarities in brain states between two different time points are represented as network edges. We use a graph-based clustering technique to identify clusters of time points representing canonical brain states, and to assess the manner in which the brain moves from one state to another as learning progresses. We observe the presence of two primary states characterized by either high activation in sensorimotor cortex or high activation in a frontal-subcortical system. Flexible switching among these primary states and other less common states becomes more frequent as learning progresses, and is inversely correlated with individual differences in learning rate. These results are consistent with the notion that the development of automaticity is associated with a greater freedom to use cognitive resources for other processes. Taken together, our work offers new insights into the constrained, low dimensional nature of brain dynamics characteristic of early learning, which give way to less constrained, high-dimensional dynamics in later learning.

Emerging Frontiers of Neuroengineering: A Network Science of Brain Connectivity.

Neuroengineering is faced with unique challenges in repairing or replacing complex neural systems that are composed of many interacting parts. These interactions form intricate patterns over large spatiotemporal scales and produce emergent behaviors that are difficult to predict from individual elements. Network science provides a particularly appropriate framework in which to study and intervene in such systems by treating neural elements (cells, volumes) as nodes in a graph and neural interactions (synapses, white matter tracts) as edges in that graph. Here, we review the emerging discipline of network neuroscience, which uses and develops tools from graph theory to better understand and manipulate neural systems from micro- to macroscales. We present examples of how human brain imaging data are being modeled with network analysis and underscore potential pitfalls. We then highlight current computational and theoretical frontiers and emphasize their utility in informing diagnosis and monitoring, brain-machine interfaces, and brain stimulation. A flexible and rapidly evolving enterprise, network neuroscience provides a set of powerful approaches and fundamental insights that are critical for the neuroengineer's tool kit.