Improved automated spot counting and modeling with bias correction.

A complete workflow was presented for estimating the concentration of microorganisms in biological samples by automatically counting spots that represent viral plaque forming units (PFU) bacterial colony forming units (CFU), or spot forming units (SFU) in images, and modeling the counts. The workflow was designed for processing images from dilution series but can also be applied to stand-alone images. The accuracy of the methods was greatly improved by adding a newly developed bias correction method. When the spots in images are densely populated, the probability of spot overlapping increases, leading to systematic undercounting. In this paper, this undercount issue was addressed in an empirical way. The proposed empirical bias correction method utilized synthetic images with known spot sizes and counts as a training set, enabling the development of an effective bias correction function using a thin-plate spline model. Its application focused on the bias correction for the automated spot counting algorithm LoST proposed by Lin et al. Simulation results demonstrated that the empirical bias correction significantly improved spot counts, reducing bias for both fixed and random spot sizes and counts.

AI/ML Models to Predict the Severity of Drug-Induced Liver Injury for Small Molecules.

Drug-induced liver injury (DILI), believed to be a multifactorial toxicity, has been a leading cause of attrition of small molecules during discovery, clinical development, and postmarketing. Identification of DILI risk early reduces the costs and cycle times associated with drug development. In recent years, several groups have reported predictive models that use physicochemical properties or in vitro and in vivo assay endpoints; however, these approaches have not accounted for liver-expressed proteins and drug molecules. To address this gap, we have developed an integrated artificial intelligence/machine learning (AI/ML) model to predict DILI severity for small molecules using a combination of physicochemical properties and off-target interactions predicted in silico. We compiled a data set of 603 diverse compounds from public databases. Among them, 164 were categorized as Most DILI (M-DILI), 245 as Less DILI (L-DILI), and 194 as No DILI (N-DILI) by the FDA. Six machine learning methods were used to create a consensus model for predicting the DILI potential. These methods include k-nearest neighbor (k-NN), support vector machine (SVM), random forest (RF), Naïve Bayes (NB), artificial neural network (ANN), logistic regression (LR), weighted average ensemble learning (WA) and penalized logistic regression (PLR). Among the analyzed ML methods, SVM, RF, LR, WA, and PLR identified M-DILI and N-DILI compounds, achieving a receiver operating characteristic area under the curve of 0.88, sensitivity of 0.73, and specificity of 0.9. Approximately 43 off-targets, along with physicochemical properties (fsp3, log S, basicity, reactive functional groups, and predicted metabolites), were identified as significant factors in distinguishing between M-DILI and N-DILI compounds. The key off-targets that we identified include: PTGS1, PTGS2, SLC22A12, PPARγ, RXRA, CYP2C9, AKR1C3, MGLL, RET, AR, and ABCC4. The present AI/ML computational approach therefore demonstrates that the integration of physicochemical properties and predicted on- and off-target biological interactions can significantly improve DILI predictivity compared to chemical properties alone.

A novel statistical test for treatment differences in clinical trials using a response-adaptive forward-looking Gittins Index Rule.

The most common objective for response-adaptive clinical trials is to seek to ensure that patients within a trial have a high chance of receiving the best treatment available by altering the chance of allocation on the basis of accumulating data. Approaches that yield good patient benefit properties suffer from low power from a frequentist perspective when testing for a treatment difference at the end of the study due to the high imbalance in treatment allocations. In this work we develop an alternative pairwise test for treatment difference on the basis of allocation probabilities of the covariate-adjusted response-adaptive randomization with forward-looking Gittins Index (CARA-FLGI) Rule for binary responses. The performance of the novel test is evaluated in simulations for two-armed studies and then its applications to multiarmed studies are illustrated. The proposed test has markedly improved power over the traditional Fisher exact test when this class of nonmyopic response adaptation is used. We also find that the test's power is close to the power of a Fisher exact test under equal randomization.

The Impact of Experimental and Calculated Error on the Performance of Affinity Predictions.

The accurate prediction of binding affinity between protein and small molecules with free energy methods, particularly the difference in binding affinities via relative binding free energy calculations, has undergone a dramatic increase in use and impact over recent years. The improvements in methodology, hardware, and implementation can deliver results with less than 1 kcal/mol mean unsigned error between calculation and experiment. This is a remarkable achievement and beckons some reflection on the significance of calculation approaching the accuracy of experiment. In this article, we describe a statistical analysis of the implications of variance (standard deviation) of both experimental and calculated binding affinities with respect to the unknown true binding affinity. We reveal that plausible ratios of standard deviation in experiment and calculation can lead to unexpected outcomes for assessing the performance of predictions. The work extends beyond the case of binding free energies to other affinity or property prediction methods.

Bayesian pooling versus sequential integration of small preclinical trials: a comparison within linear and nonlinear modeling frameworks.

Bayesian sequential integration is an appealing approach in drug development, as it allows to recursively update posterior distributions as soon as new data become available, thus considerably reducing the computation time. However, preclinical trials are often characterized by small sample sizes, which may affect the estimation process during the first integration steps, particularly when complex PK-PD models are used. In this case, sequential integration would not be practicable, and trials should be pooled together. This work is aimed at comparing simple Bayesian pooling with sequential integration through a simulation study. The two techniques are compared under several scenarios using linear as well as nonlinear models. The results of our simulation study encourage the use of Bayesian sequential integration with linear models. However, in the case of nonlinear models several caveats arise. This paper outlines some important recommendations and precautions in that respect.

Marginalized models for right-truncated and interval-censored time-to-event data.

Analysis of clustered data is often performed using random effects regression models. In such conditional models, a cluster-specific random effect is often introduced into the linear predictor function. Parameter interpretation of the covariate effects is then conditioned on the random effects, leading to a subject-specific interpretation of the regression parameters. Recently, Marginalized Multilevel Models (MMM) and the Bridge distribution models have been proposed as a unified approach, which allows one to capture the within-cluster correlations by specifying random effects while still allowing for marginal parameter interpretation. In this paper, we investigate these two approaches, and the conditional Generalized Linear Mixed Model (GLMM), in the context of right-truncated, interval-censored time-to-event data, further characterized by clustering and additional overdispersion. While these models have been applied in literature to model the mean, here we extend their application to modeling the hazard function for the survival endpoints. The models are applied to analyze data from the HET-CAMVT experiment which was designed to assess the potential of a compound to cause injection site reaction. Results show that the MMM and Bridge distribution approaches are useful when interest is in the marginal interpretation of the covariate effects.

Bayesian sequential integration within a preclinical pharmacokinetic and pharmacodynamic modeling framework: Lessons learned.

The present manuscript aims to discuss the implications of sequential knowledge integration of small preclinical trials in a Bayesian pharmacokinetic and pharmacodynamic (PK-PD) framework. While, at first sight, a Bayesian PK-PD framework seems to be a natural framework to allow for sequential knowledge integration, the scope of this paper is to highlight some often-overlooked challenges while at the same time providing some guidances in the many and overwhelming choices that need to be made. Challenges as well as opportunities will be discussed that are related to the impact of (1) the prior specification, (2) the choice of random effects, (3) the type of sequential integration method. In addition, it will be shown how the success of a sequential integration strategy is highly dependent on a carefully chosen experimental design when small trials are analyzed.

A Bayesian K-PD model for synergy: A case study.

Coadministration of 2 or more compounds can alter both the pharmacokinetics and pharmacodynamics of individual compounds. While experiments on pharmacodynamic drug-drug interactions are usually performed in an in vitro setting, this experiment focuses on an in vivo setting. The change over time of a safety biomarker is modeled using an indirect response model, in which the virtual pharmacokinetic profile of one compound drives the effect of the other. Several experiments at different dose level combinations were performed sequentially. While a traditional frequentist analysis consists of estimating the model parameters based on all the data simultaneously, in this work, we consider a Bayesian inference framework allowing to incorporate the results from a historical dose-response experiment.

Models for zero-inflated, correlated count data with extra heterogeneity: when is it too complex?

Statistical analysis of count data typically starts with a Poisson regression. However, in many real-life applications, it is observed that the variation in the counts is larger than the mean, and one needs to deal with the problem of overdispersion in the counts. Several factors may contribute to overdispersion: (1) unobserved heterogeneity due to missing covariates, (2) correlation between observations (such as in longitudinal studies), and (3) the occurrence of many zeros (more than expected from the Poisson distribution). In this paper, we discuss a model that allows one to explicitly take each of these factors into consideration. The aim of this paper is twofold: (1) investigate whether we can identify the cause of overdispersion via model selection, and (2) investigate the impact of a misspecification of the model on the power of a covariate. The paper is motivated by a study of the occurrence of drug-induced arrhythmia in beagle dogs based on electrocardiogram recordings, with the objective to evaluate the effect of potential drugs on the heartbeat irregularities. Copyright © 2016 John Wiley & Sons, Ltd.

Comparing sampling methods for pharmacokinetic studies using model averaged derived parameters.

Pharmacokinetic studies aim to study how a compound is absorbed, distributed, metabolised, and excreted. The concentration of the compound in the blood or plasma is measured at different time points after administration and pharmacokinetic parameters such as the area under the curve (AUC) or maximum concentration (Cmax ) are derived from the resulting concentration time profile. In this paper, we want to compare different methods for collecting concentration measurements (traditional sampling versus microsampling) on the basis of these derived parameters. We adjust and evaluate an existing method for testing superiority of multiple derived parameters that accounts for model uncertainty. We subsequently extend the approach to allow testing for equivalence. We motivate the methods through an illustrative example and evaluate the performance using simulations. The extensions show promising results for application to the desired setting.

A clustering algorithm for multivariate longitudinal data.

Latent growth modeling approaches, such as growth mixture models, are used to identify meaningful groups or classes of individuals in a larger heterogeneous population. But when applied to multivariate repeated measures computational problems are likely, due to the high dimension of the joint distribution of the random effects in these mixed-effects models. This article proposes a cluster algorithm for multivariate repeated data, using pseudo-likelihood and ideas based on k-means clustering, to reveal homogenous subgroups. The algorithm was demonstrated on an electro-encephalogram dataset set quantifying the effect of psychoactive compounds on the brain activity in rats.

Flexible modelling of simultaneously interval censored and truncated time-to-event data.

This paper deals with the analysis of data from a HET-CAM(VT) experiment. From a statistical perspective, such data yield many challenges. First of all, the data are typically time-to-event like data, which are at the same time interval censored and right truncated. In addition, one has to cope with overdispersion as well as clustering. Traditional analysis approaches ignore overdispersion and clustering and summarize the data into a continuous score that can be analysed using simple linear models. In this paper, a novel combined frailty model is developed that simultaneously captures all of the aforementioned statistical challenges posed by the data.

Development of a new hazard scoring system in primary neuronal cell cultures for drug-induced acute neuronal toxicity identification in early drug discovery.

We investigated drug-induced acute neuronal electrophysiological changes using Micro-Electrode arrays (MEA) to rat primary neuronal cell cultures. Data based on 6-key MEA parameters were analyzed for plate-to-plate vehicle variability, effects of positive and negative controls, as well as data from over 100 reference drugs, mostly known to have pharmacological phenotypic and clinical outcomes. A Least Absolute Shrinkage and Selection Operator (LASSO) regression, coupled with expert evaluation helped to identify the 6-key parameters from many other MEA parameters to evaluate the drug-induced acute neuronal changes. Calculating the statistical tolerance intervals for negative-positive control effects on those 4-key parameters helped us to develop a new weighted hazard scoring system on drug-induced potential central nervous system (CNS) adverse effects (AEs). The weighted total score, integrating the effects of a drug candidate on the identified six-pivotal parameters, simply determines if the testing compound/concentration induces potential CNS AEs. Hereto, it uses four different categories of hazard scores: non-neuroactive, neuroactive, hazard, or high hazard categories. This new scoring system was successfully applied to differentiate the new compounds with or without CNS AEs, and the results were correlated with the outcome of in vivo studies in mice for one internal program. Furthermore, the Random Forest classification method was used to obtain the probability that the effect of a compound is either inhibitory or excitatory. In conclusion, this new neuronal scoring system on the cell assay is actively applied in the early de-risking of drug development and reduces the use of animals and associated costs.

The influence of resolution on the predictive power of spatial heterogeneity measures as biomarkers of liver fibrosis.

Spatial heterogeneity of cells in liver biopsies can be used as biomarker for disease severity of patients. This heterogeneity can be quantified by non-parametric statistics of point pattern data, which make use of an aggregation of the point locations. The method and scale of aggregation are usually chosen ad hoc, despite values of the aforementioned statistics being heavily dependent on them. Moreover, in the context of measuring heterogeneity, increasing spatial resolution will not endlessly provide more accuracy. The question then becomes how changes in resolution influence heterogeneity indicators, and subsequently how they influence their predictive abilities. In this paper, cell level data of liver biopsy tissue taken from chronic Hepatitis B patients is used to analyze this issue. Firstly, Morisita-Horn indices, Shannon indices and Getis-Ord statistics were evaluated as heterogeneity indicators of different types of cells, using multiple resolutions. Secondly, the effect of resolution on the predictive performance of the indices in an ordinal regression model was investigated, as well as their importance in the model. A simulation study was subsequently performed to validate the aforementioned methods. In general, for specific heterogeneity indicators, a downward trend in predictive performance could be observed. While for local measures of heterogeneity a smaller grid-size is outperforming, global measures have a better performance with medium-sized grids. In addition, the use of both local and global measures of heterogeneity is recommended to improve the predictive performance.

A Bayesian, generalized frailty model for comet assays.

This paper proposes a flexible modeling approach for so-called comet assay data regularly encountered in preclinical research. While such data consist of non-Gaussian outcomes in a multilevel hierarchical structure, traditional analyses typically completely or partly ignore this hierarchical nature by summarizing measurements within a cluster. Non-Gaussian outcomes are often modeled using exponential family models. This is true not only for binary and count data, but also for, example, time-to-event outcomes. Two important reasons for extending this family are for (1) the possible occurrence of overdispersion, meaning that the variability in the data may not be adequately described by the models, which often exhibit a prescribed mean-variance link, and (2) the accommodation of a hierarchical structure in the data, owing to clustering in the data. The first issue is dealt with through so-called overdispersion models. Clustering is often accommodated through the inclusion of random subject-specific effects. Though not always, one conventionally assumes such random effects to be normally distributed. In the case of time-to-event data, one encounters, for example, the gamma frailty model (Duchateau and Janssen, 2007 ). While both of these issues may occur simultaneously, models combining both are uncommon. Molenberghs et al. ( 2010 ) proposed a broad class of generalized linear models accommodating overdispersion and clustering through two separate sets of random effects. Here, we use this method to model data from a comet assay with a three-level hierarchical structure. Although a conjugate gamma random effect is used for the overdispersion random effect, both gamma and normal random effects are considered for the hierarchical random effect. Apart from model formulation, we place emphasis on Bayesian estimation. Our proposed method has an upper hand over the traditional analysis in that it (1) uses the appropriate distribution stipulated in the literature; (2) deals with the complete hierarchical nature; and (3) uses all information instead of summary measures. The fit of the model to the comet assay is compared against the background of more conventional model fits. Results indicate the toxicity of 1,2-dimethylhydrazine dihydrochloride at different dose levels (low, medium, and high).

Measures of spatial heterogeneity in the liver tissue micro-environment as predictive factors for fibrosis score.

The organization and interaction between hepatocytes and other hepatic non-parenchymal cells plays a pivotal role in maintaining normal liver function and structure. Although spatial heterogeneity within the tumor micro-environment has been proven to be a fundamental feature in cancer progression, the role of liver tissue topology and micro-environmental factors in the context of liver damage in chronic infection has not been widely studied yet. We obtained images from 110 core needle biopsies from a cohort of chronic hepatitis B patients with different fibrosis stages according to METAVIR score. The tissue sections were immunofluorescently stained and imaged to determine the locations of CD45 positive immune cells and HBsAg-negative and HBsAg-positive hepatocytes within the tissue. We applied several descriptive techniques adopted from ecology, including Getis-Ord, the Shannon Index and the Morisita-Horn Index, to quantify the extent to which immune cells and different types of liver cells co-localize in the tissue biopsies. Additionally, we modeled the spatial distribution of the different cell types using a joint log-Gaussian Cox process and proposed several features to quantify spatial heterogeneity. We then related these measures to the patient fibrosis stage by using a linear discriminant analysis approach. Our analysis revealed that the co-localization of HBsAg-negative hepatocytes with immune cells and the co-localization of HBsAg-positive hepatocytes with immune cells are equally important factors for explaining the METAVIR score in chronic hepatitis B patients. Moreover, we found that if we allow for an error of 1 on the METAVIR score, we are able to reach an accuracy of around 80%. With this study we demonstrate how methods adopted from ecology and applied to the liver tissue micro-environment can be used to quantify heterogeneity and how these approaches can be valuable in biomarker analyses for liver topology.

Automated Spot Counting in Microbiology.

Biological samples are routinely analyzed for microbe concentration. The samples are diluted, loaded onto established host cell cultures, and incubated. If infectious agents are present in the samples, they form circular spots that do not contain the host cells. Each spot is assumed to be originated from a single microbial unit such as a bacterial colony forming unit or viral plaque forming unit. The undiluted sample concentration is estimated by counting the spots and back-calculating. Counting the number of spots by trained technicians is currently the gold standard but it is laborious, subjective, and hard to scale. This paper presents a new automated algorithm for spot counting, Localized and Sequential Thresholding (LoST). Validation studies showed that LoST performance was comparable with manual counting and outperformed several existing tools on images with overlapping spots. The LoST algorithm employs sequential thresholding through a two-stage segmentation and borrows information across all images from the same dilution series to fine-tune the count and identify right censoring. The algorithm increases the efficiency of the spot counting and the quality of the downstream analysis, especially when coupled with an appropriate statistical serial dilution model to enhance the undiluted sample concentration estimation procedure.

Joint modeling of hierarchically clustered and overdispersed non-gaussian continuous outcomes for comet assay data.

Multivariate longitudinal or clustered data are commonly encountered in clinical trials and toxicological studies. Typically, there is no single standard endpoint to assess the toxicity or efficacy of the compound of interest, but co-primary endpoints are available to assess the toxic effects or the working of the compound. Modeling the responses jointly is thus appealing to draw overall inferences using all responses and to capture the association among the responses. Non-Gaussian outcomes are often modeled univariately using exponential family models. To accommodate both the overdispersion and hierarchical structure in the data, Molenberghs et al. A family of generalized linear models for repeated measures with normal and conjugate random effects. Statistical Science 2010; 25:325-347 proposed using two separate sets of random effects. This papers considers a model for multivariate data with hierarchically clustered and overdispersed non-Gaussian data. Gamma random effect for the over-dispersion and normal random effects for the clustering in the data are being used. The two outcomes are jointly analyzed by assuming that the normal random effects for both endpoints are correlated. The association structure between the response is analytically derived. The fit of the joint model to data from a so-called comet assay are compared with the univariate analysis of the two outcomes.

An assessment of the statistical methods used to analyse toxicology studies.

The Statisticians in the Pharmaceutical Industry Toxicology Special Interest Group has collated and compared statistical analysis methods for a number of toxicology study types including general toxicology, genetic toxicology, safety pharmacology and carcinogenicity. In this paper, we present the study design, experimental units and analysis methods.

Recommendations on the statistical analysis of the Comet assay.

In 2010, the Statisticians in the Pharmaceutical Industry (PSI) Toxicology Special Interest Group met to discuss the design and analysis of the Comet assay. The Comet assay is one potential component of the package of safety studies required by regulatory bodies. As these studies usually involve a three-way nested experimental design and as the distribution of the measured response is usually either lognormal or lognormal plus a point mass at zero, the analysis is not straightforward. This has led to many different types of analysis being proposed in the literature, with several different methods applied within the pharmaceutical industry itself. This article summarises the PSI Toxicology Group's discussions and recommendations around these issues.