Optimal experimental design and estimation for q-space trajectory imaging.

Tensor-valued diffusion encoding facilitates data analysis by q-space trajectory imaging. By modeling the diffusion signal of heterogeneous tissues with a diffusion tensor distribution (DTD) and modulating the encoding tensor shape, this novel approach allows disentangling variations in diffusivity from microscopic anisotropy, orientation dispersion, and mixtures of multiple isotropic diffusivities. To facilitate the estimation of the DTD parameters, a parsimonious acquisition scheme coupled with an accurate and precise estimation of the DTD is needed. In this work, we create two precision-optimized acquisition schemes: one that maximizes the precision of the raw DTD parameters, and another that maximizes the precision of the scalar measures derived from the DTD. The improved precision of these schemes compared to a naïve sampling scheme is demonstrated in both simulations and real data. Furthermore, we show that the weighted linear least squares (WLLS) estimator that uses the squared reciprocal of the noisy signal as weights can be biased, whereas the iteratively WLLS estimator with the squared reciprocal of the predicted signal as weights outperforms the conventional unweighted linear LS and nonlinear LS estimators in terms of accuracy and precision. Finally, we show that the use of appropriate constraints can considerably increase the precision of the estimator with only a limited decrease in accuracy.

Controversial stimuli: Pitting neural networks against each other as models of human cognition.

Distinct scientific theories can make similar predictions. To adjudicate between theories, we must design experiments for which the theories make distinct predictions. Here we consider the problem of comparing deep neural networks as models of human visual recognition. To efficiently compare models' ability to predict human responses, we synthesize controversial stimuli: images for which different models produce distinct responses. We applied this approach to two visual recognition tasks, handwritten digits (MNIST) and objects in small natural images (CIFAR-10). For each task, we synthesized controversial stimuli to maximize the disagreement among models which employed different architectures and recognition algorithms. Human subjects viewed hundreds of these stimuli, as well as natural examples, and judged the probability of presence of each digit/object category in each image. We quantified how accurately each model predicted the human judgments. The best-performing models were a generative analysis-by-synthesis model (based on variational autoencoders) for MNIST and a hybrid discriminative-generative joint energy model for CIFAR-10. These deep neural networks (DNNs), which model the distribution of images, performed better than purely discriminative DNNs, which learn only to map images to labels. None of the candidate models fully explained the human responses. Controversial stimuli generalize the concept of adversarial examples, obviating the need to assume a ground-truth model. Unlike natural images, controversial stimuli are not constrained to the stimulus distribution models are trained on, thus providing severe out-of-distribution tests that reveal the models' inductive biases. Controversial stimuli therefore provide powerful probes of discrepancies between models and human perception.

The ventral striatum dissociates information expectation, reward anticipation, and reward receipt.

Do dopaminergic reward structures represent the expected utility of information similarly to a reward? Optimal experimental design models from Bayesian decision theory and statistics have proposed a theoretical framework for quantifying the expected value of information that might result from a query. In particular, this formulation quantifies the value of information before the answer to that query is known, in situations where payoffs are unknown and the goal is purely epistemic: That is, to increase knowledge about the state of the world. Whether and how such a theoretical quantity is represented in the brain is unknown. Here we use an event-related functional MRI (fMRI) task design to disentangle information expectation, information revelation and categorization outcome anticipation, and response-contingent reward processing in a visual probabilistic categorization task. We identify a neural signature corresponding to the expectation of information, involving the left lateral ventral striatum. Moreover, we show a temporal dissociation in the activation of different reward-related regions, including the nucleus accumbens, medial prefrontal cortex, and orbitofrontal cortex, during information expectation versus reward-related processing.

Optimal criteria and their asymptotic form for data selection in data-driven reduced-order modelling with Gaussian process regression.

We derive criteria for the selection of datapoints used for data-driven reduced-order modelling and other areas of supervised learning based on Gaussian process regression (GPR). While this is a well-studied area in the fields of active learning and optimal experimental design, most criteria in the literature are empirical. Here we introduce an optimality condition for the selection of a new input defined as the minimizer of the distance between the approximated output probability density function (pdf) of the reduced-order model and the exact one. Given that the exact pdf is unknown, we define the selection criterion as the supremum over the unit sphere of the native Hilbert space for the GPR. The resulting selection criterion, however, has a form that is difficult to compute. We combine results from GPR theory and asymptotic analysis to derive a computable form of the defined optimality criterion that is valid in the limit of small predictive variance. The derived asymptotic form of the selection criterion leads to convergence of the GPR model that guarantees a balanced distribution of data resources between probable and large-deviation outputs, resulting in an effective way of sampling towards data-driven reduced-order modelling. This article is part of the theme issue 'Data-driven prediction in dynamical systems'.

A model-based approach to designing developmental toxicology experiments using sea urchin embryos.

The key aim of this paper is to suggest a more quantitative approach to designing a dose-response experiment, and more specifically, a concentration-response experiment. The work proposes a departure from the traditional experimental design to determine a dose-response relationship in a developmental toxicology study. It is proposed that a model-based approach to determine a dose-response relationship can provide the most accurate statistical inference for the underlying parameters of interest, which may be estimating one or more model parameters or pre-specified functions of the model parameters, such as lethal dose, at maximal efficiency. When the design criterion or criteria can be determined at the onset, there are demonstrated efficiency gains using a more carefully selected model-based optimal design as opposed to an ad-hoc empirical design. As an illustration, a model-based approach was theoretically used to construct efficient designs for inference in a developmental toxicity study of sea urchin embryos exposed to trimethoprim. This study compares and contrasts the results obtained using model-based optimal designs versus an ad-hoc empirical design.

Optimization of asymmetric reduction conditions of 1-(benzo [d] [1,3] dioxol-5-yl) ethanone by Lactobacillus fermentum P1 using D-optimal experimental design-based model.

The biocatalytic asymmetric reduction of prochiral ketones is a significant transformation in organic chemistry as chiral carbinols are biologically active molecules and may be used as precursors of many drugs. In this study, the bioreduction of 1-(benzo [d] [1,3] dioxol-5-yl) ethanone for the production of enantiomerically pure (S)-1-(1,3-benzodioxal-5-yl) ethanol was investigated using freeze-dried whole-cell of Lactobacillus fermentum P1 and the reduction conditions was optimized with a D-optimal experimental design-based optimization methodology. This is the first study using this optimization methodology in a biocatalytic asymmetric reduction. Using D-optimal experimental design-based optimization, optimum reaction conditions were predicted as pH 6.20, temperature 30 °C, incubation time 30 h, and agitation speed 193 rpm. For these operating conditions, it was estimated that the product could be obtained with 94% enantiomeric excess (ee) and 95% conversion rate (cr). Besides, the actual ee and cr were found to be 99% tested under optimized reaction conditions. These findings demonstrated that L. fermentum P1 as an effective biocatalyst to obtain (S)-1-(1,3-benzodioxal-5-yl) ethanol and with the D-optimal experimental design-based optimization, this product could be obtained with the 99% ee and 99% cr. Finally, the proposed mathematical optimization technique showed the applicability of the obtained results for asymmetric reduction reactions.

Examination of optimized protocols for pCASL: Sensitivity to macrovascular contamination, flow dispersion, and prolonged arterial transit time.

PURPOSE: Previously, multi- post-labeling delays (PLD) pseudo-continuous arterial spin labeling (pCASL) protocols have been optimized for the estimation accuracy of the cerebral blood flow (CBF) with/without the arterial transit time (ATT) under a standard kinetic model and a normal ATT range. This study aims to examine the estimation errors of these protocols under the effects of macrovascular contamination, flow dispersion, and prolonged arrival times, all of which might differ substantially in elderly or pathological groups. METHODS: Simulated data for four protocols with varying degrees of arterial blood volume (aBV), flow dispersion, and ATTs were fitted with different kinetic models, both with and without explicit correction for macrovascular signal contamination (MVC), to obtain CBF and ATT estimates. Sensitivity to MVC was defined and calculated when aBV > 0.5%. A previously acquired dataset was retrospectively analyzed to compare with simulation. RESULTS: All protocols showed underestimation of CBF and ATT in the prolonged ATT range. With MVC, the protocol optimized for CBF only (CBFopt) had the lowest sensitivity value to MVC, 33.47% and 60.21% error per 1% aBV in simulation and in vivo, respectively, among multi-PLD protocols. All multi-PLD protocols showed a significant decrease in estimation error when an extended kinetic model was used. Increasing flow dispersion at short ATTs caused increasing CBF and ATT overestimation in all protocols. CONCLUSION: CBFopt was the least sensitive protocol to prolonged ATT and MVC for CBF estimation while maintaining reasonably good performance in estimating ATT. Explicitly including a macrovascular component in the kinetic model was shown to be a feasible approach in controlling for MVC.

Data-driven experimental design and model development using Gaussian process with active learning.

Interest in computational modeling of cognition and behavior continues to grow. To be most productive, modelers should be equipped with tools that ensure optimal efficiency in data collection and in the integrity of inference about the phenomenon of interest. Traditionally, models in cognitive science have been parametric, which are particularly susceptible to model misspecification because their strong assumptions (e.g. parameterization, functional form) may introduce unjustified biases in data collection and inference. To address this issue, we propose a data-driven nonparametric framework for model development, one that also includes optimal experimental design as a goal. It combines Gaussian Processes, a stochastic process often used for regression and classification, with active learning, from machine learning, to iteratively fit the model and use it to optimize the design selection throughout the experiment. The approach, dubbed Gaussian process with active learning (GPAL), is an extension of the parametric, adaptive design optimization (ADO) framework (Cavagnaro, Myung, Pitt, & Kujala, 2010). We demonstrate the application and features of GPAL in a delay discounting task and compare its performance to ADO in two experiments. The results show that GPAL is a viable modeling framework that is noteworthy for its high sensitivity to individual differences, identifying novel patterns in the data that were missed by the model-constrained ADO. This investigation represents a first step towards the development of a data-driven cognitive modeling framework that serves as a middle ground between raw data, which can be difficult to interpret, and parametric models, which rely on strong assumptions.

ADOpy: a python package for adaptive design optimization.

Experimental design is fundamental to research, but formal methods to identify good designs are lacking. Advances in Bayesian statistics and machine learning offer algorithm-based ways to identify good experimental designs. Adaptive design optimization (ADO; Cavagnaro, Myung, Pitt, & Kujala, 2010; Myung, Cavagnaro, & Pitt, 2013) is one such method. It works by maximizing the informativeness and efficiency of data collection, thereby improving inference. ADO is a general-purpose method for conducting adaptive experiments on the fly and can lead to rapid accumulation of information about the phenomenon of interest with the fewest number of trials. The nontrivial technical skills required to use ADO have been a barrier to its wider adoption. To increase its accessibility to experimentalists at large, we introduce an open-source Python package, ADOpy, that implements ADO for optimizing experimental design. The package, available on GitHub, is written using high-level modular-based commands such that users do not have to understand the computational details of the ADO algorithm. In this paper, we first provide a tutorial introduction to ADOpy and ADO itself, and then illustrate its use in three walk-through examples: psychometric function estimation, delay discounting, and risky choice. Simulation data are also provided to demonstrate how ADO designs compare with other designs (random, staircase).

Identification of the relative timing of infectiousness and symptom onset for outbreak control.

In an outbreak of an emerging disease the epidemiological characteristics of the pathogen may be largely unknown. A key determinant of ability to control the outbreak is the relative timing of infectiousness and symptom onset. We provide a method for identifying this relationship with high accuracy based on data from simulated household-stratified symptom-onset data. Further, this can be achieved with observations taken on only a few specific days, chosen optimally, within each household. The information provided by this method may inform decision making processes for outbreak response. An accurate and computationally-efficient heuristic for determining the optimal surveillance scheme is introduced. This heuristic provides a novel approach to optimal design for Bayesian model discrimination.

Selective recruitment designs for improving observational studies using electronic health records.

Large-scale electronic health records (EHRs) present an opportunity to quickly identify suitable individuals in order to directly invite them to participate in an observational study. EHRs can contain data from millions of individuals, raising the question of how to optimally select a cohort of size n from a larger pool of size N. In this article, we propose a simple selective recruitment protocol that selects a cohort in which covariates of interest tend to have a uniform distribution. We show that selectively recruited cohorts potentially offer greater statistical power and more accurate parameter estimates than randomly selected cohorts. Our protocol can be applied to studies with multiple categorical and continuous covariates. We apply our protocol to a numerically simulated prospective observational study using an EHR database of stable acute coronary disease patients from 82 089 individuals in the U.K. Selective recruitment designs require a smaller sample size, leading to more efficient and cost-effective studies.

Subset selection via continuous optimization with applications to network design.

Choosing a subset of representative items from a set of alternatives is an important problem in many scientific fields such as environmental science and statistics. For most practical problems, however, a computationally efficient solution method is not known to exist. While this problem has attracted a significant amount of attention, the majority of specifically designed algorithms do not scale well with respect to the problem size or do not provide a usable open-source package. In this study, we show that any global continuous optimization technique can be used for solving the representative subset selection problem. The latter is achieved by designing a simple transformation which embeds the problem's discrete space into a larger continuous space. The proposed methodology is applied to design problems in environmental and statistical domains. We evaluate the proposed method using several open-source global optimization packages, and show that this technique compares favorably with existing direct methods.

Gaussian Process Based Expected Information Gain Computation for Bayesian Optimal Design.

Optimal experimental design (OED) is of great significance in efficient Bayesian inversion. A popular choice of OED methods is based on maximizing the expected information gain (EIG), where expensive likelihood functions are typically involved. To reduce the computational cost, in this work, a novel double-loop Bayesian Monte Carlo (DLBMC) method is developed to efficiently compute the EIG, and a Bayesian optimization (BO) strategy is proposed to obtain its maximizer only using a small number of samples. For Bayesian Monte Carlo posed on uniform and normal distributions, our analysis provides explicit expressions for the mean estimates and the bounds of their variances. The accuracy and the efficiency of our DLBMC and BO based optimal design are validated and demonstrated with numerical experiments.

A general framework for optimizing arterial spin labeling MRI experiments.

PURPOSE: Arterial spin labeling (ASL) MRI is a non-invasive perfusion imaging technique that is inherently SNR limited, so scan protocols ideally need to be rigorously optimized to provide the most accurate measurements. A general framework is presented for optimizing ASL experiments to achieve optimal accuracy for perfusion estimates and, if required, other hemodynamic parameters, within a fixed scan time. The effectiveness of this framework is then demonstrated by optimizing the post-labeling delays (PLDs) of a multi-PLD pseudo-continuous ASL experiment and validating the improvement using simulations and in vivo data. THEORY AND METHODS: A simple framework is proposed based on the use of the Cramér-Rao lower bound to find the protocol design which minimizes the predicted parameter estimation errors. Protocols were optimized for cerebral blood flow (CBF) accuracy or both CBF and arterial transit time (ATT) accuracy and compared to a conventional multi-PLD protocol, with evenly spaced PLDs, and a single-PLD protocol, using simulations and in vivo experiments in healthy volunteers. RESULTS: Simulations and in vivo data agreed extremely well with the predicted performance of all protocols. For the in vivo experiments, optimizing for just CBF resulted in a 48% and 15% decrease in CBF errors, relative to the reference multi-PLD and single-PLD protocols, respectively. Optimizing for both CBF and ATT reduced CBF errors by 37%, without a reduction in ATT accuracy, relative to the reference multi-PLD protocol. CONCLUSION: The presented framework can effectively design ASL experiments to minimize measurement errors based on the requirements of the scan.

Information-adaptive clinical trials with selective recruitment and binary outcomes.

Selective recruitment designs preferentially recruit individuals who are estimated to be statistically informative onto a clinical trial. Individuals who are expected to contribute less information have a lower probability of recruitment. Furthermore, in an information-adaptive design, recruits are allocated to treatment arms in a manner that maximises information gain. The informativeness of an individual depends on their covariate (or biomarker) values, and how information is defined is a critical element of information-adaptive designs. In this paper, we define and evaluate four different methods for quantifying statistical information. Using both experimental data and numerical simulations, we show that selective recruitment designs can offer a substantial increase in statistical power compared with randomised designs. In trials without selective recruitment, we find that allocating individuals to treatment arms according to information-adaptive protocols also leads to an increase in statistical power. Consequently, selective recruitment designs can potentially achieve successful trials using fewer recruits thereby offering economic and ethical advantages. Copyright © 2017 John Wiley & Sons, Ltd.

Optimizing long-circulating liposomes for delivery of simvastatin to C26 colon carcinoma cells.

Simvastatin (SIM) is a lipophilic statin that has potential benefits for prevention and treatment of several types of malignancies. However, its low water solubility and the toxicity associated with administration of high doses recommend it for encapsulation in carriers able to deliver the therapeutic dose in the tumor. In this work, liposomes with long-circulating properties were proposed as delivery systems for SIM. The objective of this study was to optimize the formulation of SIM-loaded long-circulating liposomes (LCL-SIM) by using D-optimal experimental design. The influence of phospholipids concentration, phospholipids to cholesterol molar ratio and SIM concentration was studied on SIM liposomal concentration, encapsulation efficiency and liposomal size. The optimized formulation had liposomal SIM concentration 6238 µg/ml, EE % of 83.4% and vesicle size of 190.5 nm. Additionally we evaluated the in vitro cytotoxicity of the optimized liposomal SIM (LCL-SIM-OPT) on C26 murine colon carcinoma cells cultivated in monoculture as well as in co-culture with murine peritoneal macrophages at a cell density ratio that provides an approximation of physiological conditions of colon carcinoma development in vivo. Our preliminary studies suggested that LCL-SIM-OPT exerted cytotoxicity on C26 cells probably via enhancement of oxidative stress in co-culture environment.

Optimal Design of Non-equilibrium Experiments for Genetic Network Interrogation.

Many experimental systems in biology, especially synthetic gene networks, are amenable to perturbations that are controlled by the experimenter. We developed an optimal design algorithm that calculates optimal observation times in conjunction with optimal experimental perturbations in order to maximize the amount of information gained from longitudinal data derived from such experiments. We applied the algorithm to a validated model of a synthetic Brome Mosaic Virus (BMV) gene network and found that optimizing experimental perturbations may substantially decrease uncertainty in estimating BMV model parameters.

Optimal experimental designs for accelerated failure time with Type I and random censoring.

Proportional Hazards models have been widely used to analyze survival data. In many cases survival data do not verify the assumption of proportional hazards. An alternative to the PH models with more relaxed conditions are Accelerated Failure Time models. These models are fairly commonly used in the field of manufacturing, but they are more and more frequent for modeling clinical trial data. They focus on the direct effect of the explanatory variables on the survival function allowing an easier interpretation of the effect of the corresponding covariates on the survival time. Optimal experimental designs are computed in this framework for Type I and random arrival. The results are applied to clinical models used to prevent tuberculosis in Ugandan adults infected with HIV.

Identifying Bayesian optimal experiments for uncertain biochemical pathway models.

Pharmacodynamic (PD) models are mathematical models of cellular reaction networks that include drug mechanisms of action. These models are useful for studying predictive therapeutic outcomes of novel drug therapies in silico. However, PD models are known to possess significant uncertainty with respect to constituent parameter data, leading to uncertainty in the model predictions. Furthermore, experimental data to calibrate these models is often limited or unavailable for novel pathways. In this study, we present a Bayesian optimal experimental design approach for improving PD model prediction accuracy. We then apply our method using simulated experimental data to account for uncertainty in hypothetical laboratory measurements. This leads to a probabilistic prediction of drug performance and a quantitative measure of which prospective laboratory experiment will optimally reduce prediction uncertainty in the PD model. The methods proposed here provide a way forward for uncertainty quantification and guided experimental design for models of novel biological pathways.

Mechanistic analysis of challenge-response experiments.

We present an application of mechanistic modeling and nonlinear longitudinal regression in the context of biomedical response-to-challenge experiments, a field where these methods are underutilized. In this type of experiment, a system is studied by imposing an experimental challenge, and then observing its response. The combination of mechanistic modeling and nonlinear longitudinal regression has brought new insight, and revealed an unexpected opportunity for optimal design. Specifically, the mechanistic aspect of our approach enables the optimal design of experimental challenge characteristics (e.g., intensity, duration). This article lays some groundwork for this approach. We consider a series of experiments wherein an isolated rabbit heart is challenged with intermittent anoxia. The heart responds to the challenge onset, and recovers when the challenge ends. The mean response is modeled by a system of differential equations that describe a candidate mechanism for cardiac response to anoxia challenge. The cardiac system behaves more variably when challenged than when at rest. Hence, observations arising from this experiment exhibit complex heteroscedasticity and sharp changes in central tendency. We present evidence that an asymptotic statistical inference strategy may fail to adequately account for statistical uncertainty. Two alternative methods are critiqued qualitatively (i.e., for utility in the current context), and quantitatively using an innovative Monte-Carlo method. We conclude with a discussion of the exciting opportunities in optimal design of response-to-challenge experiments.