Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts.

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Menstrual cycle-driven hormone concentrations co-fluctuate with white and gray matter architecture changes across the whole brain.

Cyclic fluctuations in hypothalamic-pituitary-gonadal axis (HPG-axis) hormones exert powerful behavioral, structural, and functional effects through actions on the mammalian central nervous system. Yet, very little is known about how these fluctuations alter the structural nodes and information highways of the human brain. In a study of 30 naturally cycling women, we employed multidimensional diffusion and T1-weighted imaging during three estimated menstrual cycle phases (menses, ovulation, and mid-luteal) to investigate whether HPG-axis hormone concentrations co-fluctuate with alterations in white matter (WM) microstructure, cortical thickness (CT), and brain volume. Across the whole brain, 17ß-estradiol and luteinizing hormone (LH) concentrations were directly proportional to diffusion anisotropy (µFA; 17ß-estradiol: ß1 = 0.145, highest density interval (HDI) = [0.211, 0.4]; LH: ß1 = 0.111, HDI = [0.157, 0.364]), while follicle-stimulating hormone (FSH) was directly proportional to CT (ß1 = 0 .162, HDI = [0.115, 0.678]). Within several individual regions, FSH and progesterone demonstrated opposing relationships with mean diffusivity (Diso) and CT. These regions mainly reside within the temporal and occipital lobes, with functional implications for the limbic and visual systems. Finally, progesterone was associated with increased tissue (ß1 = 0.66, HDI = [0.607, 15.845]) and decreased cerebrospinal fluid (CSF; ß1 = -0.749, HDI = [-11.604, -0.903]) volumes, with total brain volume remaining unchanged. These results are the first to report simultaneous brain-wide changes in human WM microstructure and CT coinciding with menstrual cycle-driven hormone rhythms. Effects were observed in both classically known HPG-axis receptor-dense regions (medial temporal lobe, prefrontal cortex) and in other regions located across frontal, occipital, temporal, and parietal lobes. Our results suggest that HPG-axis hormone fluctuations may have significant structural impacts across the entire brain.

Diffusion MRI head motion correction methods are highly accurate but impacted by denoising and sampling scheme.

Head motion correction is particularly challenging in diffusion-weighted MRI (dMRI) scans due to the dramatic changes in image contrast at different gradient strengths and directions. Head motion correction is typically performed using a Gaussian Process model implemented in FSL's Eddy. Recently, the 3dSHORE-based SHORELine method was introduced that does not require shell-based acquisitions, but it has not been previously benchmarked. Here we perform a comprehensive evaluation of both methods on realistic simulations of a software fiber phantom that provides known ground-truth head motion. We demonstrate that both methods perform remarkably well, but that performance can be impacted by sampling scheme and the extent of head motion and the denoising strategy applied before head motion correction. Furthermore, we find Eddy benefits from denoising the data first with MP-PCA. In sum, we provide the most extensive known benchmarking of dMRI head motion correction, together with extensive simulation data and a reproducible workflow. PRACTITIONER POINTS: Both Eddy and SHORELine head motion correction methods performed quite well on a large variety of simulated data. Denoising with MP-PCA can improve head motion correction performance when Eddy is used. SHORELine effectively corrects motion in non-shelled diffusion spectrum imaging data.

A practical evaluation of measures derived from compressed sensing diffusion spectrum imaging.

Diffusion Spectrum Imaging (DSI) using dense Cartesian sampling of q-space has been shown to provide important advantages for modeling complex white matter architecture. However, its adoption has been limited by the lengthy acquisition time required. Sparser sampling of q-space combined with compressed sensing (CS) reconstruction techniques has been proposed as a way to reduce the scan time of DSI acquisitions. However prior studies have mainly evaluated CS-DSI in post-mortem or non-human data. At present, the capacity for CS-DSI to provide accurate and reliable measures of white matter anatomy and microstructure in the living human brain remains unclear. We evaluated the accuracy and inter-scan reliability of 6 different CS-DSI schemes that provided up to 80% reductions in scan time compared to a full DSI scheme. We capitalized on a dataset of 26 participants who were scanned over eight independent sessions using a full DSI scheme. From this full DSI scheme, we subsampled images to create a range of CS-DSI images. This allowed us to compare the accuracy and inter-scan reliability of derived measures of white matter structure (bundle segmentation, voxel-wise scalar maps) produced by the CS-DSI and the full DSI schemes. We found that CS-DSI estimates of both bundle segmentations and voxel-wise scalars were nearly as accurate and reliable as those generated by the full DSI scheme. Moreover, we found that the accuracy and reliability of CS-DSI was higher in white matter bundles that were more reliably segmented by the full DSI scheme. As a final step, we replicated the accuracy of CS-DSI in a prospectively acquired dataset (n = 20, scanned once). Together, these results illustrate the utility of CS-DSI for reliably delineating in vivo white matter architecture in a fraction of the scan time, underscoring its promise for both clinical and research applications.

Stress affects navigation strategies in immersive virtual reality.

There are known individual differences in both the ability to learn the layout of novel environments and the flexibility of strategies for navigating known environments. However, it is unclear how navigational abilities are impacted by high-stress scenarios. Here we used immersive virtual reality (VR) to develop a novel behavioral paradigm to examine navigation under dynamically changing situations. We recruited 48 participants (24 female; ages 17-32) to navigate a virtual maze (7.5 m × 7.5 m). Participants learned the maze by moving along a fixed path past the maze's landmarks (paintings). Subsequently, participants experienced either a non-stress condition, or a high-stress condition tasking them with navigating the maze. In the high-stress condition, their initial path was blocked, the environment was darkened, threatening music was played, fog obstructed more distal views of the environment, and participants were given a time limit of 20 s with a countdown timer displayed at the top of their screen. On trials where the path was blocked, we found self-reported stress levels and distance traveled increased while trial completion rate decreased (as compared to non-stressed control trials). On unblocked stress trials, participants were less likely to take a shortcut and consequently navigated less efficiently compared to control trials. Participants with more trait spatial anxiety reported more stress and navigated less efficiently. Overall, our results suggest that navigational abilities change considerably under high-stress conditions.

Corticospinal premotor fibers facilitate complex motor control after stroke.

OBJECTIVE: The corticospinal tract (CST) is considered the most important motor output pathway comprising fibers from the primary motor cortex (M1) and various premotor areas. Damage to its descending fibers after stroke commonly leads to motor impairment. While premotor areas are thought to critically support motor recovery after stroke, the functional role of their corticospinal output for different aspects of post-stroke motor control remains poorly understood. METHODS: We assessed the differential role of CST fibers originating from premotor areas and M1 in the control of basal (single-joint muscle synergies and strength) and complex motor control (involving inter-joint coordination and visuomotor integration) using a novel diffusion imaging approach in chronic stroke patients. RESULTS: While M1 sub-tract anisotropy was positively correlated with basal and complex motor skills, anisotropy of PMd, PMv, and SMA sub-tracts was exclusively associated with complex motor tasks. Interestingly, patients featuring persistent motor deficits showed an additional positive association between premotor sub-tract integrity and basal motor control. INTERPRETATION: While descending M1 output seems to be a prerequisite for any form of upper limb movements, complex motor skills critically depend on output from premotor areas after stroke. The additional involvement of premotor tracts in basal motor control in patients with persistent deficits emphasizes their compensatory capacity in post-stroke motor control. In summary, our findings highlight the pivotal role of descending corticospinal output from premotor areas for motor control after stroke, which thus serve as prime candidates for future interventions to amplify motor recovery.