Planning to Learn: A Novel Algorithm for Active Learning during Model-Based Planning.

Active Inference is a recently developed framework for modeling decision processes under uncertainty. Over the last several years, empirical and theoretical work has begun to evaluate the strengths and weaknesses of this approach and how it might be extended and improved. One recent extension is the "sophisticated inference" (SI) algorithm, which improves performance on multi-step planning problems through a recursive decision tree search. However, little work to date has been done to compare SI to other established planning algorithms in reinforcement learning (RL). In addition, SI was developed with a focus on inference as opposed to learning. The present paper therefore has two aims. First, we compare performance of SI to Bayesian RL schemes designed to solve similar problems. Second, we present and compare an extension of SI - sophisticated learning (SL) - that more fully incorporates active learning during planning. SL maintains beliefs about how model parameters would change under the future observations expected under each policy. This allows a form of counterfactual retrospective inference in which the agent considers what could be learned from current or past observations given different future observations. To accomplish these aims, we make use of a novel, biologically inspired environment that requires an optimal balance between goal-seeking and active learning, and which was designed to highlight the problem structure for which SL offers a unique solution. This setup requires an agent to continually search an open environment for available (but changing) resources in the presence of competing affordances for information gain. Our simulations demonstrate that SL outperforms all other algorithms in this context - most notably, Bayes-adaptive RL and upper confidence bound (UCB) algorithms, which aim to solve multi-step planning problems using similar principles (i.e., directed exploration and counterfactual reasoning about belief updates given different possible actions/observations). These results provide added support for the utility of Active Inference in solving this class of biologically-relevant problems and offer added tools for testing hypotheses about human cognition.

A flexible framework for anomaly Detection via dimensionality reduction.

Anomaly detection is challenging, especially for large datasets in high dimensions. Here, we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. DRAMA is released as a general python package that implements the general framework with a wide range of built-in options. This approach identifies the primary prototypes in the data with anomalies detected by their large distances from the prototypes, either in the latent space or in the original, high-dimensional space. DRAMA is tested on a wide variety of simulated and real datasets, in up to 3000 dimensions, and is found to be robust and highly competitive with commonly used anomaly detection algorithms, especially in high dimensions. The flexibility of the DRAMA framework allows for significant optimization once some examples of anomalies are available, making it ideal for online anomaly detection, active learning, and highly unbalanced datasets. Besides, DRAMA naturally provides clustering of outliers for subsequent analysis.

Cytisine versus nicotine for smoking cessation.

BACKGROUND: Placebo-controlled trials indicate that cytisine, a partial agonist that binds the nicotinic acetylcholine receptor and is used for smoking cessation, almost doubles the chances of quitting at 6 months. We investigated whether cytisine was at least as effective as nicotine-replacement therapy in helping smokers to quit. METHODS: We conducted a pragmatic, open-label, noninferiority trial in New Zealand in which 1310 adult daily smokers who were motivated to quit and called the national quitline were randomly assigned in a 1:1 ratio to receive cytisine for 25 days or nicotine-replacement therapy for 8 weeks. Cytisine was provided by mail, free of charge, and nicotine-replacement therapy was provided through vouchers for low-cost patches along with gum or lozenges. Low-intensity, telephone-delivered behavioral support was provided to both groups through the quitline. The primary outcome was self-reported continuous abstinence at 1 month. RESULTS: At 1 month, continuous abstinence from smoking was reported for 40% of participants receiving cytisine (264 of 655) and 31% of participants receiving nicotine-replacement therapy (203 of 655), for a difference of 9.3 percentage points (95% confidence interval, 4.2 to 14.5). The effectiveness of cytisine for continuous abstinence was superior to that of nicotine-replacement therapy at 1 week, 2 months, and 6 months. In a prespecified subgroup analysis of the primary outcome, cytisine was superior to nicotine-replacement therapy among women and noninferior among men. Self-reported adverse events over 6 months occurred more frequently in the cytisine group (288 events among 204 participants) than in the group receiving nicotine-replacement therapy (174 events among 134 participants); adverse events were primarily nausea and vomiting and sleep disorders. CONCLUSIONS: When combined with brief behavioral support, cytisine was found to be superior to nicotine-replacement therapy in helping smokers quit smoking, but it was associated with a higher frequency of self-reported adverse events. (Funded by the Health Research Council of New Zealand; Australian New Zealand Clinical Trials Registry number, ACTRN12610000590066.).

A general test of the Copernican principle.

To date, there has been no general way of determining if the Copernican principle--that we live at a typical position in the Universe--is in fact a valid assumption, significantly weakening the foundations of cosmology as a scientific endeavor. Here we present an observational test for the Copernican assumption which can be automatically implemented while we search for dark energy in the coming decade. Our test is entirely independent of any model for dark energy or theory of gravity and thereby represents a model-independent test of the Copernican principle.

Dark energy reflections in the redshift-space quadrupole.

We show that the redshift-space quadrupole will be a powerful tool for constraining dark energy even if the baryon oscillations are missing from the monopole power spectrum and bias is scale and time dependent. We calculate the accuracy with which next-generation galaxy surveys such as KAOS will measure the quadrupole power spectrum, which gives the leading anisotropies in the power spectrum in redshift space due to linear velocity, and the so-called "Finger of God" and Alcock-Paczynski effects. Combining the monopole and quadrupole power spectra, in the complete absence of baryon oscillations (Omegab=0), leads to a roughly 500% improvement in constraints on dark energy compared with those from the monopole spectrum alone.