Planning to Learn: A Novel Algorithm for Active Learning during Model-Based Planning.

Active Inference is a recently developed framework for modeling decision processes under uncertainty. Over the last several years, empirical and theoretical work has begun to evaluate the strengths and weaknesses of this approach and how it might be extended and improved. One recent extension is the "sophisticated inference" (SI) algorithm, which improves performance on multi-step planning problems through a recursive decision tree search. However, little work to date has been done to compare SI to other established planning algorithms in reinforcement learning (RL). In addition, SI was developed with a focus on inference as opposed to learning. The present paper therefore has two aims. First, we compare performance of SI to Bayesian RL schemes designed to solve similar problems. Second, we present and compare an extension of SI - sophisticated learning (SL) - that more fully incorporates active learning during planning. SL maintains beliefs about how model parameters would change under the future observations expected under each policy. This allows a form of counterfactual retrospective inference in which the agent considers what could be learned from current or past observations given different future observations. To accomplish these aims, we make use of a novel, biologically inspired environment that requires an optimal balance between goal-seeking and active learning, and which was designed to highlight the problem structure for which SL offers a unique solution. This setup requires an agent to continually search an open environment for available (but changing) resources in the presence of competing affordances for information gain. Our simulations demonstrate that SL outperforms all other algorithms in this context - most notably, Bayes-adaptive RL and upper confidence bound (UCB) algorithms, which aim to solve multi-step planning problems using similar principles (i.e., directed exploration and counterfactual reasoning about belief updates given different possible actions/observations). These results provide added support for the utility of Active Inference in solving this class of biologically-relevant problems and offer added tools for testing hypotheses about human cognition.

Correlation Constraints for Regression Models: Controlling Bias in Brain Age Prediction.

In neuroimaging, the difference between chronological age and predicted brain age, also known as brain age delta, has been proposed as a pathology marker linked to a range of phenotypes. Brain age delta is estimated using regression, which involves a frequently observed bias due to a negative correlation between chronological age and brain age delta. In brain age prediction models, this correlation can manifest as an overprediction of the age of young brains and an underprediction for elderly ones. We show that this bias can be controlled for by adding correlation constraints to the model training procedure. We develop an analytical solution to this constrained optimization problem for Linear, Ridge, and Kernel Ridge regression. The solution is optimal in the least-squares sense i.e., there is no other model that satisfies the correlation constraints and has a better fit. Analyses on the PAC2019 competition data demonstrate that this approach produces optimal unbiased predictive models with a number of advantages over existing approaches. Finally, we introduce regression toolboxes for Python and MATLAB that implement our algorithm.

Structural neuroimaging biomarkers for obsessive-compulsive disorder in the ENIGMA-OCD consortium: medication matters.

No diagnostic biomarkers are available for obsessive-compulsive disorder (OCD). Here, we aimed to identify magnetic resonance imaging (MRI) biomarkers for OCD, using 46 data sets with 2304 OCD patients and 2068 healthy controls from the ENIGMA consortium. We performed machine learning analysis of regional measures of cortical thickness, surface area and subcortical volume and tested classification performance using cross-validation. Classification performance for OCD vs. controls using the complete sample with different classifiers and cross-validation strategies was poor. When models were validated on data from other sites, model performance did not exceed chance-level. In contrast, fair classification performance was achieved when patients were grouped according to their medication status. These results indicate that medication use is associated with substantial differences in brain anatomy that are widely distributed, and indicate that clinical heterogeneity contributes to the poor performance of structural MRI as a disease marker.

Outcome prediction with serial neuron-specific enolase and machine learning in anoxic-ischaemic disorders of consciousness.

BACKGROUND: The continuation of life-sustaining therapy in critical care patients with anoxic-ischemic disorders of consciousness (AI-DOC) depends on prognostic tests such as serum neuron-specific enolase (NSE) concentration levels. OBJECTIVES: To apply predictive models using machine learning methods to examine, one year after onset, the prognostic power of serial measurements of NSE in patients with AI-DOC. To compare the discriminative accuracy of this method to both standard single-day, absolute, and difference-between-days, relative NSE levels. METHODS: Classification algorithms were implemented and K-nearest neighbours (KNN) imputation was used to avoid complete case elimination of patients with missing NSE values. Non-imputed measurements from Day 0 to Day 6 were used for single day and difference-between-days. RESULTS: The naive Bayes classifier on imputed serial NSE measurements returned an AUC of (0.81±0.07) for n=126 patients (100 poor outcome). This was greater than logistic regression (0.73±0.08) and all other classifiers. Naive Bayes gave a specificity and sensitivity of 96% and 49%, respectively, for an (uncalibrated) probability decision threshold of 90%. The maximum AUC for a single day was Day 3 (0.75) for a subset of n=79 (61 poor outcome) patients, and for differences between Day 1 and Day 4 (0.81) for a subset of n=46 (39 poor outcome) patients. CONCLUSION: Imputation avoided the elimination of patients with missing data and naive Bayes outperformed all other classifiers. Machine learning algorithms could detect automatically discriminatory features and the overall predictive power increased from standard methods due to the larger data set. CODE AVAILABILITY: Data analysis code is available under GNU at: https://github.com/emilymuller1991/outcome_prediction_nse.

Consciousness Indexing and Outcome Prediction with Resting-State EEG in Severe Disorders of Consciousness.

We applied the following methods to resting-state EEG data from patients with disorders of consciousness (DOC) for consciousness indexing and outcome prediction: microstates, entropy (i.e. approximate, permutation), power in alpha and delta frequency bands, and connectivity (i.e. weighted symbolic mutual information, symbolic transfer entropy, complex network analysis). Patients with unresponsive wakefulness syndrome (UWS) and patients in a minimally conscious state (MCS) were classified into these two categories by fitting and testing a generalised linear model. We aimed subsequently to develop an automated system for outcome prediction in severe DOC by selecting an optimal subset of features using sequential floating forward selection (SFFS). The two outcome categories were defined as UWS or dead, and MCS or emerged from MCS. Percentage of time spent in microstate D in the alpha frequency band performed best at distinguishing MCS from UWS patients. The average clustering coefficient obtained from thresholding beta coherence performed best at predicting outcome. The optimal subset of features selected with SFFS consisted of the frequency of microstate A in the 2-20 Hz frequency band, path length obtained from thresholding alpha coherence, and average path length obtained from thresholding alpha coherence. Combining these features seemed to afford high prediction power. Python and MATLAB toolboxes for the above calculations are freely available under the GNU public license for non-commercial use ( https://qeeg.wordpress.com ).