Peptide set test: a peptide-centric strategy to infer differentially expressed proteins.

MOTIVATION: The clinical translation of mass spectrometry-based proteomics has been challenging due to limited statistical power caused by large technical variability and inter-patient heterogeneity. Bottom-up proteomics provides an indirect measurement of proteins through digested peptides. This raises the question whether peptide measurements can be used directly to better distinguish differentially expressed proteins. RESULTS: We present a novel method called the peptide set test, which detects coordinated changes in the expression of peptides originating from the same protein and compares them to the rest of the peptidome. Applying our method to data from a published spike-in experiment and simulations demonstrates improved sensitivity without compromising precision, compared to aggregation-based approaches. Additionally, applying the peptide set test to compare the tumor proteomes of tamoxifen-sensitive and tamoxifen-resistant breast cancer patients reveals significant alterations in peptide levels of collagen XII, suggesting an association between collagen XII-mediated matrix reassembly and tamoxifen resistance. Our study establishes the peptide set test as a powerful peptide-centric strategy to infer differential expression in proteomics studies. AVAILABILITY AND IMPLEMENTATION: Peptide set test (PepSetTest) is publicly available at https://github.com/JmWangBio/PepSetTest.

DOSE-L1000: unveiling the intricate landscape of compound-induced transcriptional changes.

MOTIVATION: The LINCS L1000 project has collected gene expression profiles for thousands of compounds across a wide array of concentrations, cell lines, and time points. However, conventional analysis methods often fall short in capturing the rich information encapsulated within the L1000 transcriptional dose-response data. RESULTS: We present DOSE-L1000, a database that unravels the potency and efficacy of compound-gene pairs and the intricate landscape of compound-induced transcriptional changes. Our study uses the fitting of over 140 million generalized additive models and robust linear models, spanning the complete spectrum of compounds and landmark genes within the LINCS L1000 database. This systematic approach provides quantitative insights into differential gene expression and the potency and efficacy of compound-gene pairs across diverse cellular contexts. Through examples, we showcase the application of DOSE-L1000 in tasks such as cell line and compound comparisons, along with clustering analyses and predictions of drug-target interactions. DOSE-L1000 fosters applications in drug discovery, accelerating the transition to omics-driven drug development. AVAILABILITY AND IMPLEMENTATION: DOSE-L1000 is publicly available at https://doi.org/10.5281/zenodo.8286375.

Non-constant mean relative potency for antibody-dependent cellular cytotoxicity assays.

Bioassays are regulated, analytical methods used to ensure proper activity (potency) of biological products at release and during long-term storage. Potency is commonly reported on a relative basis by comparing and calibrating a concentration-response curve from the test material to that of a reference standard material. The relative potency approach depends on an assumption that the two concentration-response curves exhibit similar (equivalent) shapes, except for a potency shift. In certain circumstances, however, biological factors preclude the similarity assumption, and the traditional approach becomes unworkable. The antibody-mediated cytotoxicity assay is one example where the similarity assumption does not always hold. Other examples also arise in the fields of toxicology and pharmacology. In this work, we present a non-constant mean relative potency approach which averages the relative potency across a common range of the concentration-response curves. The proposed method captures the changing nature of the relative potency into a summary statistic that can be reported for batch calibration and quality control purposes. We provide inferential methods for this statistic and summarize the results of a simulation comparing these methods across a number of non-constant relative potency scenarios and assay conditions.

Strategy for Designing In Vivo Dose-Response Comparison Studies.

In preclinical drug discovery, at the step of lead optimization of a compound, in vivo experimentation can differentiate several compounds in terms of efficacy and potency in a biological system of whole living organisms. For the lead optimization study, it may be desirable to implement a dose-response design so that compound comparisons can be made from nonlinear curves fitted to the data. A dose-response design requires more thought relative to a simpler study design, needing parameters for the number of doses, the dose values, and the sample size per dose. This tutorial illustrates how to calculate statistical power, choose doses, and determine sample size per dose for a comparison of two or more dose-response curves for a future in vivo study.

Quality by Design for Preclinical In Vitro Assay Development.

Quality by Design (QbD) is an approach to assay development to determine the design space, which is the range of assay variable settings that should result in satisfactory assay quality. Typically, QbD is applied in manufacturing, but it works just as well in the preclinical space. Through three examples, we illustrate the QbD approach with experimental design and associated data analysis to determine the design space for preclinical assays.

Whole-cage randomization for animal studies with unequal cage or group sizes.

Following good statistical practice, in vivo study investigators allocate animals into two or more treatment groups using a randomization routine to eliminate selection bias and balance known and unknown confounding factors. For some studies, however, randomization at the individual animal level cannot be implemented. For example, for studies that involve co-housed male mice, an animal-level randomization can place unfamiliar mice together in the same cage, which can trigger fighting. To meet the ethical obligations to enhance the welfare of an animal used in science, the experimental procedures are, therefore, often modified, and male mice, possibly from the same brood, may be housed together. It follows that animal allocation into groups must proceed at the whole-cage level. Given the small sample sizes in animal studies, controlling baseline variables can be quite challenging. The difficulty greatly increases with a whole-cage randomization restriction. When the number of animals per cage or the treatment group sizes are unequal, there is no algorithm in the literature to perform the task. We propose a novel, fast, and reliable algorithm to provide a whole-cage randomization that balances one or more baseline variables across groups. The algorithm was applied to a realistic example dataset.

A comparison of statistical methods for animal oncology studies.

In pre-clinical oncology studies, tumor-bearing animals are treated and observed over a period of time in order to measure and compare the efficacy of one or more cancer-intervention therapies along with a placebo/standard of care group. A data analysis is typically carried out by modeling and comparing tumor volumes, functions of tumor volumes, or survival. Data analysis on tumor volumes is complicated because animals under observation may be euthanized prior to the end of the study for one or more reasons, such as when an animal's tumor volume exceeds an upper threshold. In such a case, the tumor volume is missing not-at-random for the time remaining in the study. To work around the non-random missingness issue, several statistical methods have been proposed in the literature, including the rate of change in log tumor volume and partial area under the curve. In this work, an examination and comparison of the test size and statistical power of these and other popular methods for the analysis of tumor volume data is performed through realistic Monte Carlo computer simulations. The performance, advantages, and drawbacks of popular statistical methods for animal oncology studies are reported. The recommended methods are applied to a real data set.

Mean comparisons and power calculations to ensure reproducibility in preclinical drug discovery.

In the pharmaceutical industry, in vivo animal experiments are conducted to test the effects of novel preclinical drug compounds. Well-planned animal studies involve a sample size and statistical power analysis to provide a basis for the number of animals allocated into comparator arms of a future study. These calculations require approximate values for the parameters of a statistical model that will be applied to the future data and used to test for differences via statistical hypotheses. If the prestudy parameter estimates are nearly correct, the power analysis guarantees that a difference will be detected from the study data, up to a prespecified probability. Traditional power computations, however, are not calculated with reproducibility in mind. In this work, the issue of reproducibility in drug discovery is tackled from the point of view that study-to-study variability is not included in a typical sample size and power analysis. Three proposed methods that yield a reproducible mean-comparison analysis are derived and compared.

The current state of Bayesian methods in nonclinical pharmaceutical statistics: Survey results and recommendations from the DIA/ASA-BIOP Nonclinical Bayesian Working Group.

The use of Bayesian methods to support pharmaceutical product development has grown in recent years. In clinical statistics, the drive to provide faster access for patients to medical treatments has led to a heightened focus by industry and regulatory authorities on innovative clinical trial designs, including those that apply Bayesian methods. In nonclinical statistics, Bayesian applications have also made advances. However, they have been embraced far more slowly in the nonclinical area than in the clinical counterpart. In this article, we explore some of the reasons for this slower rate of adoption. We also present the results of a survey conducted for the purpose of understanding the current state of Bayesian application in nonclinical areas and for identifying areas of priority for the DIA/ASA-BIOP Nonclinical Bayesian Working Group. The survey explored current usage, hurdles, perceptions, and training needs for Bayesian methods among nonclinical statisticians. Based on the survey results, a set of recommendations is provided to help guide the future advancement of Bayesian applications in nonclinical pharmaceutical statistics.

Comparisons of outlier tests for potency bioassays.

Potency bioassays are used to measure biological activity. Consequently, potency is considered a critical quality attribute in manufacturing. Relative potency is measured by comparing the concentration-response curves of a manufactured test batch with that of a reference standard. If the curve shapes are deemed similar, the test batch is said to exhibit constant relative potency with the reference standard, a critical requirement for calibrating the potency of the final drug product. Outliers in bioassay potency data may result in the false acceptance/rejection of a bad/good sample and, if accepted, may yield a biased relative potency estimate. To avoid these issues, the USP<1032> recommends the screening of bioassay data for outliers prior to performing a relative potency analysis. In a recently published work, the effects of one or more outliers, outlier size, and outlier type on similarity testing and estimation of relative potency were thoroughly examined, confirming the USP<1032> outlier guidance. As a follow-up, several outlier detection methods, including those proposed by the USP<1010>, are evaluated and compared in this work through computer simulation. Two novel outlier detection methods are also proposed. The effects of outlier removal on similarity testing and estimation of relative potency were evaluated, resulting in recommendations for best practice.

A fast and reliable test for parallelism in bioassay.

Parallelism in bioassay is a synonym of similarity between two concentration-response curves. Before the determination of relative potency in bioassays, it is necessary to test for and claim parallelism between the pair of concentration-response curves of reference standard and test sample. Methods for parallelism testing include p-value-based significance tests and interval-based equivalence tests. Most of the latter approaches make statistical inference about the equivalence of parameters of the concentration-response curve models. An apparent drawback of such methods is that equivalence in model parameters does not guarantee similarity between the reference and test sample. In contrast, a Bayesian method was recently proposed that directly tests the parallelism hypothesis that the concentration-response curve of the test sample is a horizontal shift of that of the reference. In other words, the testing sample is a dilution or concentration of the reference standard. The Bayesian approach is shown to protect against type I error and provides sufficient statistical power for parallelism testing. In practice, however, it is challenging to implement the method as it requires both specialized Bayesian software and a relatively long run time. In this paper, we propose a frequentist version of the test with split-second run time. The empirical properties of the frequentist parallelism test method are evaluated and compared with the original Bayesian method. It is demonstrated that the frequentist method is both fast and reliable for parallelism testing for a variety of concentration-response models.

Detecting bliss synergy in in vivo combination studies with a tumor kinetic model.

Linear models are generally reliable methods for analyzing tumor growth in vivo, with drug effectiveness being represented by the steepness of the regression slope. With immunotherapy, however, not all tumor growth follows a linear pattern, even after log transformation. Tumor kinetics models are mechanistic models that describe tumor proliferation and tumor killing macroscopically, through a set of differential equations. In drug combination studies, although an additional drug-drug interaction term can be added to such models, however, the drug interactions suggested by tumor kinetics models cannot be translated directly into synergistic effects. We have developed a novel statistical approach that simultaneously models tumor growth in control, monotherapy, and combination therapy groups. This approach makes it possible to test for synergistic effects directly and to compare such effects among different studies.

Content uniformity testing for stratified samples via parametric tolerance interval testing.

Historically in the biopharmaceutical setting, USP<905> has been used to establish that a batch of drug product has acceptable content uniformity. More recently, alternative approaches such as the two one-sided parametric tolerance interval test (PTI-TOST) have been proposed to establish content uniformity. Traditionally, the PTI-TOST is implemented as a sequential, two-tiered test, under the generally accepted assumption that the data are independently and identically distributed. Since the material is sequenced through the manufacturing process over a period of time, there are conceptually arguable locations within each batch, for instance: beginning, middle, and end. In such a situation, a practitioner may wish to evaluate potential effects of these batch locations, for example, during process validation. If location (stratified) differences exist within the batch and if multiple samples are taken from each location, significant within-location correlations may be induced in the data. In such a case, the traditional PTI-TOST underestimates the total variability, thereby improperly boosting the power of the test method. When there is reason to believe that location variances exist, the batch may be evaluated using stratified sampling, and the location effect may be modeled. In this paper, a two-tiered PTI-TOST that accounts for both between-location and within-location variance components is introduced. Operating characteristic curves and practical advice are given to aid the practitioner's uptake of the proposed method.

Effect of a statistical outlier in potency bioassays.

The USP<1032> guidelines recommend the screening of bioassay data for outliers prior to performing a relative potency (RP) analysis. The guidelines, however, do not offer advice on the size or type of outlier that should be removed prior to model fitting and calculation of RP. Computer simulation was used to investigate the consequences of ignoring the USP<1032> guidance to remove outliers. For biotherapeutics and vaccines, outliers in potency data may result in the false acceptance/rejection of a bad/good lot of drug product. Biological activity, measured through a potency bioassay, is considered a critical quality attribute in manufacturing. If the concentration-response potency curve of a test sample is deemed to be similar in shape to that of the reference standard, the curves are said to exhibit constant RP, an essential criterion for the interpretation of a RP. One or more outliers in the concentration-response data, however, may result in a failure to declare similarity or may yield a biased RP estimate. Concentration-response curves for test and reference were computer generated with constant RP from four-parameter logistic curves. Single outlier, multiple outlier, and whole-curve outlier scenarios were explored for their effects on the similarity testing and on the RP estimation. Though the simulations point to situations for which outlier removal is unnecessary, the results generally support the USP<1032> recommendation and illustrate the impact on the RP calculation when application of outlier removal procedures are discounted.

Pharmacokinetics/Pharmacodynamics of Peptide Deformylase Inhibitor GSK1322322 against Streptococcus pneumoniae, Haemophilus influenzae, and Staphylococcus aureus in Rodent Models of Infection.

GSK1322322 is a novel inhibitor of peptide deformylase (PDF) with good in vitro activity against bacteria associated with community-acquired pneumonia and skin infections. We have characterized the in vivo pharmacodynamics (PD) of GSK1322322 in immunocompetent animal models of infection with Streptococcus pneumoniae and Haemophilus influenzae (mouse lung model) and with Staphylococcus aureus (rat abscess model) and determined the pharmacokinetic (PK)/PD index that best correlates with efficacy and its magnitude. Oral PK studies with both models showed slightly higher-than-dose-proportional exposure, with 3-fold increases in area under the concentration-time curve (AUC) with doubling doses. GSK1322322 exhibited dose-dependent in vivo efficacy against multiple isolates of S. pneumoniae, H. influenzae, and S. aureus. Dose fractionation studies with two S. pneumoniae and S. aureus isolates showed that therapeutic outcome correlated best with the free AUC/MIC (fAUC/MIC) index in S. pneumoniae (R(2), 0.83), whereas fAUC/MIC and free maximum drug concentration (fCmax)/MIC were the best efficacy predictors for S. aureus (R(2), 0.9 and 0.91, respectively). Median daily fAUC/MIC values required for stasis and for a 1-log10 reduction in bacterial burden were 8.1 and 14.4 for 11 S. pneumoniae isolates (R(2), 0.62) and 7.2 and 13.0 for five H. influenzae isolates (R(2), 0.93). The data showed that for eight S. aureus isolates, fAUC correlated better with efficacy than fAUC/MIC (R(2), 0.91 and 0.76, respectively), as efficacious AUCs were similar for all isolates, independent of their GSK1322322 MIC (range, 0.5 to 4 µg/ml). Median fAUCs of 2.1 and 6.3 µg · h/ml were associated with stasis and 1-log10 reductions, respectively, for S. aureus.

A new PK equivalence test for a bridging study.

In a bridging study, the plasma drug concentration-time curve is generally used to assess bioequivalence between the two formulations. Selected pharmacokinetic (PK) parameters including the area under the concentration-time curve, the maximum plasma concentration or peak exposure (Cmax), and drug half-life (T1/2) are compared to ensure comparable bioavailability of the two formulations. Comparability in these PK parameters, however, does not necessarily imply equivalence of the entire concentration-time profile. In this article, we propose an alternative metric of equivalence based on the maximum difference between PK profiles of the two formulations. A test procedure based on Bayesian analysis and accounting for uncertainties in model parameters is developed. Through both theoretical derivation and empirical simulation, it is shown that the new method provides better control over consumer's risk.

Nilotinib versus imatinib as first-line therapy for patients with unresectable or metastatic gastrointestinal stromal tumours (ENESTg1): a randomised phase 3 trial.

BACKGROUND: Nilotinib inhibits the tyrosine kinase activity of ABL1/BCR-ABL1 and KIT, platelet-derived growth factor receptors (PDGFRs), and the discoidin domain receptor. Gain-of-function mutations in KIT or PDGFRα are key drivers in most gastrointestinal stromal tumours (GISTs). This trial was designed to test the efficacy and safety of nilotinib versus imatinib as first-line therapy for patients with advanced GISTs. METHODS: In this randomised, open-label, multicentre, phase 3 trial (ENESTg1), participants from academic centres were aged 18 years or older and had previously untreated, histologically confirmed, metastatic or unresectable GISTs. Patients were stratified by previous adjuvant therapy and randomly assigned (1:1) via a randomisation list to receive oral imatinib 400 mg once daily or oral nilotinib 400 mg twice daily. The primary endpoint was centrally reviewed progression-free survival. Efficacy endpoints were assessed by intention-to-treat. This trial is registered with ClinicalTrials.gov, number NCT00785785. FINDINGS: Because the futility boundary was crossed at a preplanned interim analysis, trial accrual terminated in April, 2011. Between March 16, 2009, and April 21, 2011, 647 patients were enrolled; of whom 324 were allocated nilotinib and 320 were allocated imatinib. At final analysis of the core study (data cutoff, October, 2012), 2-year progression-free survival was higher in the imatinib group (59·2% [95% CI 50·9-66·5]) than in the nilotinib group (51·6% [43·0-59·5]; hazard ratio 1·47 [95% CI 1·10-1·95]). In the imatinib group, the most common grade 3-4 adverse events were hypophosphataemia (19 [6%]), anaemia (17 [5%]), abdominal pain (13; 4%), and elevated lipase level (15; 5%), and in the nilotinib group were anaemia (18; 6%), elevated lipase level (15; 5%), elevated alanine aminotransferase concentration (12; 4%), and abdominal pain (11; 3%). The most common serious adverse event in both groups was abdominal pain (11 [4%] in the imatinib group, 14 [4%] in the nilotinib group). INTERPRETATION: Nilotinib cannot be recommended for broad use to treat first-line GIST. However, future studies might identify patient subsets for whom first-line nilotinib could be of clinical benefit. FUNDING: Novartis Pharmaceuticals.

Testing assay linearity over a pre-specified range.

Validation of linearity is a regulatory requirement. Although many methods are proposed, they suffer from several deficiencies including difficulties of setting fit-for-purpose acceptable limits, dependency on concentration levels used in linearity experiment, and challenges in implementation for statistically lay users. In this article, a statistical procedure for testing linearity is proposed. The method uses a two one-sided test (TOST) of equivalence to evaluate the bias that can result from approximating a higher-order polynomial response with a linear function. By using orthogonal polynomials and generalized pivotal quantity analysis, the method provides a closed-form solution, thus making linearity testing easy to implement.

Dissolution curve comparisons through the F(2) parameter, a Bayesian extension of the f(2) statistic.

Dissolution (or in vitro release) studies constitute an important aspect of pharmaceutical drug development. One important use of such studies is for justifying a biowaiver for post-approval changes which requires establishing equivalence between the new and old product. We propose a statistically rigorous modeling approach for this purpose based on the estimation of what we refer to as the F2 parameter, an extension of the commonly used f2 statistic. A Bayesian test procedure is proposed in relation to a set of composite hypotheses that capture the similarity requirement on the absolute mean differences between test and reference dissolution profiles. Several examples are provided to illustrate the application. Results of our simulation study comparing the performance of f2 and the proposed method show that our Bayesian approach is comparable to or in many cases superior to the f2 statistic as a decision rule. Further useful extensions of the method, such as the use of continuous-time dissolution modeling, are considered.

Testing drug additivity based on monotherapies.

Under the Loewe additivity, constant relative potency between two drugs is a sufficient condition for the two drugs to be additive. Implicit in this condition is that one drug acts like a dilution of the other. Geometrically, it means that the dose-response curve of one drug is a copy of another that is shifted horizontally by a constant over the log-dose axis. Such phenomenon is often referred to as parallelism. Thus, testing drug additivity is equivalent to the demonstration of parallelism between two dose-response curves. Current methods used for testing parallelism are usually based on significance tests for differences between parameters in the dose-response curves of the monotherapies. A p-value of less than 0.05 is indicative of non-parallelism. The p-value-based methods, however, may be fundamentally flawed because an increase in either sample size or precision of the assay used to measure drug effect may result in more frequent rejection of parallel lines for a trivial difference. Moreover, similarity (difference) between model parameters does not necessarily translate into the similarity (difference) between the two response curves. As a result, a test may conclude that the model parameters are similar (different), yet there is little assurance on the similarity between the two dose-response curves. In this paper, we introduce a Bayesian approach to directly test the hypothesis that the two drugs have a constant relative potency. An important utility of our proposed method is in aiding go/no-go decisions concerning two drug combination studies. It is illustrated with both a simulated example and a real-life example.