Statistical Testing for Protein Equivalence Identifies Core Functional Modules Conserved across 360 Cancer Cell Lines and Presents a General Approach to Investigating Biological Systems.

Quantitative proteomics has enhanced our capability to study protein dynamics and their involvement in disease using various techniques, including statistical testing, to discern the significant differences between conditions. While most focus is on what is different between conditions, exploring similarities can provide valuable insights. However, exploring similarities directly from the analyte level, such as proteins, genes, or metabolites, is not a standard practice and is not widely adopted. In this study, we propose a statistical framework called QuEStVar (Quantitative Exploration of Stability and Variability through statistical hypothesis testing), enabling the exploration of quantitative stability and variability of features with a combined statistical framework. QuEStVar utilizes differential and equivalence testing to expand statistical classifications of analytes when comparing conditions. We applied our method to an extensive data set of cancer cell lines and revealed a quantitatively stable core proteome across diverse tissues and cancer subtypes. The functional analysis of this set of proteins highlighted the molecular mechanism of cancer cells to maintain constant conditions of the tumorigenic environment via biological processes, including transcription, translation, and nucleocytoplasmic transport.

Equivalence Testing Based Fit Index: Standardized Root Mean Squared Residual.

A popular measure of model fit in structural equation modeling (SEM) is the standardized root mean squared residual (SRMR) fit index. Equivalence testing has been used to evaluate model fit in structural equation modeling (SEM) but has yet to be applied to SRMR. Accordingly, the present study proposed equivalence-testing based fit tests for the SRMR (ESRMR). Several variations of ESRMR were introduced, incorporating different equivalence bounds and methods of computing confidence intervals. A Monte Carlo simulation study compared these novel tests with traditional methods for evaluating model fit. The results demonstrated that certain ESRMR tests based on an analytic computation of the confidence interval correctly reject poor-fitting models and are well-powered for detecting good-fitting models. We also present an illustrative example with real data to demonstrate how ESRMR may be incorporated into model fit evaluation and reporting. Our recommendation is that ESRMR tests be presented in addition to descriptive fit indices for model fit reporting in SEM.

[Comparison of DNA Extraction Methods for Processed Foods Containing Soybean or Maize].

Processed foods containing soybean or maize are subject to labeling regulations pertinent to genetically modified (GM) foods in Japan. To confirm the reliability of the labeling procedure of GM foods, the Japanese standard analytical methods (standard methods) using real-time PCR technique have been established. Although certain DNA extraction protocols are stipulated as standard in these methods, the use of other protocols confirmed to be equivalent to the existing ones was permitted. In this study, the equivalence testing of the techniques employed for DNA extraction from processed foods containing soybean or corn was conducted. In this study, the equivalence testing of the techniques employed for DNA extraction from processed foods containing soybean or maize was conducted. The silica membrane-based DNA extraction kits, GM quicker 4 and DNeasy Plant Maxi Kit (Maxi Kit), as an existing method were compared. GM quicker 4 was considered to be equivalent to or better than Maxi Kit.

Comparative safety assessment of genetically modified crops: focus on equivalence with reference varieties could contribute to more efficient and effective field trials.

The initial compositional analysis of plants plays an important role within the internationally harmonized comparative safety assessment approach for genetically modified plants. Current EFSA guidance prescribes two types of comparison, namely difference tests with regard to a conventional comparator or control, and equivalence tests with regard to a collection of commercial reference varieties. The experience gained so far shows that most of the statistically significant differences between the test and control can be discounted based on the fact that they are still within equivalence limits of reference varieties with a presumed history of safe use. Inclusion of a test variety and reference varieties into field trial design, and of the statistical equivalence test would already suffice for the purpose of finding relevant parameters that warrant further assessment, hence both the inclusion of a conventional counterpart and the performance of difference testing can be omitted. This would also allow for the inclusion of safety testing regimes into plant variety testing VCU (value for cultivation and use) or other, independent variety trials.

External validation of a shortened screening tool using individual participant data meta-analysis: A case study of the Patient Health Questionnaire-Dep-4.

Shortened versions of self-reported questionnaires may be used to reduce respondent burden. When shortened screening tools are used, it is desirable to maintain equivalent diagnostic accuracy to full-length forms. This manuscript presents a case study that illustrates how external data and individual participant data meta-analysis can be used to assess the equivalence in diagnostic accuracy between a shortened and full-length form. This case study compares the Patient Health Questionnaire-9 (PHQ-9) and a 4-item shortened version (PHQ-Dep-4) that was previously developed using optimal test assembly methods. Using a large database of 75 primary studies (34,698 participants, 3,392 major depression cases), we evaluated whether the PHQ-Dep-4 cutoff of ≥ 4 maintained equivalent diagnostic accuracy to a PHQ-9 cutoff of ≥ 10. Using this external validation dataset, a PHQ-Dep-4 cutoff of ≥ 4 maximized the sum of sensitivity and specificity, with a sensitivity of 0.88 (95% CI 0.81, 0.93), 0.68 (95% CI 0.56, 0.78), and 0.80 (95% CI 0.73, 0.85) for the semi-structured, fully structured, and MINI reference standard categories, respectively, and a specificity of 0.79 (95% CI 0.74, 0.83), 0.85 (95% CI 0.78, 0.90), and 0.83 (95% CI 0.80, 0.86) for the semi-structured, fully structured, and MINI reference standard categories, respectively. While equivalence with a PHQ-9 cutoff of ≥ 10 was not established, we found the sensitivity of the PHQ-Dep-4 to be non-inferior to that of the PHQ-9, and the specificity of the PHQ-Dep-4 to be marginally smaller than the PHQ-9.

Equivalence tests before end of follow-up under the class of log transformation model.

One important topic in clinical trials is to show that the effects of new and standard treatments are equivalent in terms of clinical relevance. In literature, many equivalence tests based on the maximal difference between two survival functions for the two treatments over the whole time axis have been proposed. However, since survival times can only be observed until the end of follow-up, an equivalence test should be based on a comparison only in the observed time-window dictated by the end of follow-up. In this article, under the class of log transformation model, we propose an asymptotical α-level equivalence test for the difference between two survival functions that only addresses equivalence until the end of follow-up. We demonstrate that the hypothesis of equivalence of two survival functions before the end of follow-up can be formulated as interval-based hypothesis testing which involves the treatment effect parameter. Simulation results indicate that when sample size is sufficiently large the proposed test controls the type I error effectively and performs well at detecting the equivalence. The proposed test is applied to a dataset from veteran's administration lung cancer trial.

How to Choose between Different Bayesian Posterior Indices for Hypothesis Testing in Practice.

Hypothesis testing is an essential statistical method in experimental psychology and the cognitive sciences. The problems of traditional null hypothesis significance testing (NHST) have been discussed widely, and among the proposed solutions to the replication problems caused by the inappropriate use of significance tests and p-values is a shift toward Bayesian data analysis. However, Bayesian hypothesis testing is concerned with various posterior indices for significance and the size of an effect. This complicates Bayesian hypothesis testing in practice, as the availability of multiple Bayesian alternatives to the traditional p-value causes confusion which one to select and why. In this paper, various Bayesian posterior indices which have been proposed in the literature are compared and their benefits and limitations are discussed. The comparison shows that conceptually not all proposed Bayesian alternatives to NHST and p-values are beneficial, and the usefulness of some indices strongly depends on the study design and research goal. However, the comparison also reveals that there exist at least two candidates among the available Bayesian posterior indices which have appealing theoretical properties and are widely underused in the cognitive sciences.

Is "not different" enough to conclude similar cardiovascular responses across sexes?

The number of research studies investigating whether similar or different cardiovascular responses or adaptations exist between males and females is increasing. Traditionally, difference-based statistical methods, e.g., t test, ANOVA, etc., have been implemented to compare cardiovascular function between males and females, with a P value of >0.05 used to denote similarity between sexes. However, an absence of evidence, i.e., large P value, is not evidence of absence, i.e., no sex differences. Equivalence testing determines whether two measures or groups provide statistically equivalent outcomes, in that they differ by less than an "ideally prespecified" smallest effect size of interest. Our perspective discusses the applicability and utility of integrating equivalence testing when conducting sex comparisons in cardiovascular research. An emphasis is placed on how cardiovascular researchers may conduct equivalence testing across multiple study designs, e.g., cross-sectional comparisons, repeated-measures intervention, etc. The strengths and weaknesses of this statistical tool are discussed. Equivalence analyses are relatively simple to conduct, may be used in conjunction with traditional hypothesis testing to interpret findings, and permit the determination of statistically equivalent responses between sexes. We recommend that cardiovascular researchers consider implementing equivalence testing to better our understanding of similar and different cardiovascular processes between sexes.

The effects of U.S. county and state income inequality on self-reported happiness and health are equivalent to zero.

PURPOSE: A popular idea in the social sciences is that contexts with high income inequality undermine people's well-being and health. However, existing studies documenting this phenomenon typically compare a small number of higher-level units (countries/regions). Here, we use local income inequality indicators and temporal designs to provide the most highly powered test to date of the associations between income inequality and self-reported happiness and health in the USA METHOD: We combined county-level income inequality data (county-level Gini coefficients) with the responses from the General Social Survey (GSS) Cross-sectional dataset (13,000 + participants from ≈1000 county-waves) and Panels (3 × 3000 + participants from 3 × ≈500 county-waves); we used the GSS happiness ("not too happy," "pretty happy," or "very happy") and health ("poor," "fair," "good," or "excellent") variables. RESULTS: Multilevel-ordered logistic models and equivalence tests revealed that the within-county effects of income inequality on self-reported happiness and health were systematically equivalent to zero. Additional analyses revealed that the within-state effects were identical, that using alternative measures of state income inequality led to the same conclusions, and that lagged effects (between + 1 and + 12 years) were never significant and always equivalent to zero. CONCLUSION: The present work suggests that-at least in the USA-income inequality is likely neither associated with self-reported happiness nor with self-reported health.

Equivalence testing of a newly developed interviewer-led telephone script for the EORTC QLQ-C30.

PURPOSE: The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life-Core Questionnaire (QLQ-C30) is a widely used generic self-report measure of health-related quality of life (HRQOL) for cancer patients. However, no validated voice script for interviewer-led telephone administration was previously available. The aim of this study was to develop a voice script for interviewer administration via telephone. METHODS: Following guidelines from the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) PRO Mixed Modes Good Research Practices Task Force, a randomised cross-over equivalence study, including cognitive debriefing, was conducted to assess equivalence between paper and telephone administration modes. Assuming an expected intraclass correlation coefficient (ICC) of 0.70 and a minimally acceptable level of 0.50, a sample size of 63 was required. RESULTS: Cognitive interviews with five cancer patients found the voice script to be clear and understandable. Due to a protocol deviation in the first wave of testing, only 26 patients were available for analyses. A second wave of recruitment was conducted, adding 37 patients (n = 63; mean age 55.48; 65.1% female). Total ICCs for mode comparison ranged from 0.72 (nausea and vomiting, 95% CI 0.48-0.86) to 0.90 (global health status/QoL, 95% CI 0.80-0.95; pain, 95% CI 0.79-0.95; constipation, 95% CI 0.80-0.95). For paper first administration, all ICCs were above 0.70, except nausea and vomiting (ICC 0.55; 95% CI 0.24-0.76) and financial difficulties (ICC 0.60; 95% CI 0.31-0.79). For phone first administration, all ICCs were above 0.70. CONCLUSIONS: The equivalence testing results support the voice script's validity for administration of the QLQ-C30 via telephone.

Smoothed Quantiles for χ2 Type Test Statistics with Applications.

Chi-square type test statistics are widely used in assessing the goodness-of-fit of a theoretical model. The exact distributions of such statistics can be quite different from the nominal chi-square distribution due to violation of conditions encountered with real data. In such instances, the bootstrap or Monte Carlo methodology might be used to approximate the distribution of the statistic. However, the sample quantile may be a poor estimate of the population counterpart when either the sample size is small or the number of different values of the replicated statistic is limited. Using statistical learning, this article develops a method that yields more accurate quantiles for chi-square type test statistics. Formulas for smoothing the quantiles of chi-square type statistics are obtained. Combined with the bootstrap methodology, the smoothed quantiles are further used to conduct equivalence testing in mean and covariance structure analysis. Two real data examples illustrate the applications of the developed formulas in quantifying the size of model misspecification under equivalence testing. The idea developed in the article can also be used to develop formulas for smoothing the quantiles of other types of test statistics or parameter estimates.

Shared genomic segment analysis with equivalence testing.

An important aspect of disease gene mapping is replication, that is, a putative finding in one group of individuals is confirmed in another set of individuals. As it can happen by chance that individuals share an estimated disease position, we developed a statistical approach to determine the p-value for multiple individuals or families to share a possibly small number of candidate susceptibility variants. Here, we focus on candidate variants for dominant traits that have been obtained by our previously developed heterozygosity analysis, and we are testing the sharing of candidate variants obtained for different individuals. Our approach allows for multiple pathogenic variants in a gene to contribute to disease, and for estimated disease variant positions to be imprecise. Statistically, the method developed here falls into the category of equivalence testing, where the classical null and alternative hypotheses of homogeneity and heterogeneity are reversed. The null hypothesis situation is created by permuting genomic locations of variants for one individual after another. We applied our methodology to the ALSPAC data set of 1,927 whole-genome sequenced individuals, where some individuals carry a pathogenic variant for the BRCA1 gene, but no two individuals carry the same variant. Our shared genomic segment analysis found significant evidence for BRCA1 pathogenic variants within ±5 kb of a given DNA variant.

Bayesian Hodges-Lehmann tests for statistical equivalence in the two-sample setting: Power analysis, type I error rates and equivalence boundary selection in biomedical research.

BACKGROUND: Null hypothesis significance testing (NHST) is among the most frequently employed methods in the biomedical sciences. However, the problems of NHST and p-values have been discussed widely and various Bayesian alternatives have been proposed. Some proposals focus on equivalence testing, which aims at testing an interval hypothesis instead of a precise hypothesis. An interval hypothesis includes a small range of parameter values instead of a single null value and the idea goes back to Hodges and Lehmann. As researchers can always expect to observe some (although often negligibly small) effect size, interval hypotheses are more realistic for biomedical research. However, the selection of an equivalence region (the interval boundaries) often seems arbitrary and several Bayesian approaches to equivalence testing coexist. METHODS: A new proposal is made how to determine the equivalence region for Bayesian equivalence tests based on objective criteria like type I error rate and power. Existing approaches to Bayesian equivalence testing in the two-sample setting are discussed with a focus on the Bayes factor and the region of practical equivalence (ROPE). A simulation study derives the necessary results to make use of the new method in the two-sample setting, which is among the most frequently carried out procedures in biomedical research. RESULTS: Bayesian Hodges-Lehmann tests for statistical equivalence differ in their sensitivity to the prior modeling, power, and the associated type I error rates. The relationship between type I error rates, power and sample sizes for existing Bayesian equivalence tests is identified in the two-sample setting. Results allow to determine the equivalence region based on the new method by incorporating such objective criteria. Importantly, results show that not only can prior selection influence the type I error rate and power, but the relationship is even reverse for the Bayes factor and ROPE based equivalence tests. CONCLUSION: Based on the results, researchers can select between the existing Bayesian Hodges-Lehmann tests for statistical equivalence and determine the equivalence region based on objective criteria, thus improving the reproducibility of biomedical research.

Sameness and Difference in Psychological Research on Consensually Non-Monogamous Relationships: The Need for Invariance and Equivalence Testing.

Comparative research involving consensually non-monogamous (CNM) relationships and outcomes related to well-being continues to grow as an area of interest within sexual science. However, claims of sameness and/or difference between groups rely on two critical, yet widely under-appreciated assumptions: that the concepts being compared between groups are the same (i.e., measurement invariance), and that logically and statistically coherent procedures are used for evaluating sameness (i.e., equivalence testing). We evaluated the state of measurement invariance and equivalence across three studies, involving different types of CNM comparisons (i.e., relationship types, partner types) and designs (analysis of primary individual data, primary dyadic data, and secondary data). Our invariance tests of CNM compared to monogamous individuals (Study 1) and "primary" compared to "secondary" partners in dyadic appraisal of CNM individuals (Study 2) revealed that many measures of well-being failed to replicate their measurement models and were not generalizable across relationship types or partner types. Our reanalyses of existing comparative CNM effects using individual and meta-analyzed equivalence tests (Study 3), meanwhile, indicated that this literature requires more consistent reporting practices and larger samples, as most studies produced uninformative tests of equivalence. Our results illustrate the importance of auxiliary hypothesis evaluation and statistical procedure selection for generating informative comparative tests. Our findings also highlight potential divergences in social construction of well-being. We offer suggestions for researchers, reviewers, and editors in terms of needed methodological reforms for future comparative CNM research.

Improving the criterion validity of the activPAL in determining physical activity intensity during laboratory and free-living conditions.

The activPAL is a valid measure of step counts and posture, but its ability to determine physical activity intensity is unclear. This study tested the criterion validity of the activPAL using its built-in linear cadence-metabolic equivalents (METs) equation (activPAL-linear) versus an individualized height-adjusted curvilinear cadence-METs equation (activPAL-curvilinear) to estimate intensity-related physical activity. Forty adults (25±6 years, 23.3±4.1 kg/m2) wore an activPAL during a 7-stage progressive treadmill walking protocol (criterion: indirect calorimetry). A sub-sample (n=32) wore the device during free-living conditions for 7-days (criterion: PiezoRxD monitor). In the laboratory, the activPAL-linear overestimated METs during slow walking (1.5-3.0 miles•hour-1) but underestimated METs during fast walking (3.5-4.5 miles•hour-1) (all, p<0.001). In the free-living condition, the activPAL-linear overestimated time in light-intensity activity and underestimated moderate-intensity activity (both, p<0.001), but did not register any vigorous-intensity activity. In contrast, the activPAL-curvilinear estimated values statistically equivalent to indirect calorimetry for treadmill stages 1-6 (1.5-4.0 miles•hour-1) and to the PiezoRxD determined light- and moderate-intensity activity during free-living. We present a simple, data processing technique that uses an alternative curvilinear cadence-MET equation that improves the ability of the activPAL to measure intensity-related physical activity in both laboratory and free-living settings.

The evaluation of Rolimeter, KLT, KiRA and KT-1000 arthrometer in healthy individuals shows acceptable intra-rater but poor inter-rater reliability in the measurement of anterior tibial knee translation.

PURPOSE: To assess measurement equivalence, inter- and intra-rater reliability, standard error of measurements (SEM) and false positive measurements (FPM) of four different knee arthrometers (KLT,Karl Storz; KiRA, I + ; KT-1000 MEDmetric Corp; Rolimeter, Aircast) in healthy patients. METHODS: Four different investigators (two advanced (AR) and two beginners (BR)) examined 12 participants with healthy knees at two time points with regards to anterior tibial translation (ATT) and side-to-side difference (SSD). Test equivalence was assessed using the TOST (two-one-sided t test) procedure with ± 1 mm equivalence boundaries. Intraclass correlation coefficients (ICCs) were calculated using two-way mixed effects models. Furthermore, false positive-(SSD > 3 mm) and SEMs were assessed. RESULTS: A total of 2304 Lachman Tests were performed. Between-rater SSDs were equivalent between AR and BR raters for the Rolimeter only. Inter-rater ICC values (SSD, ATT) were graded as "poor" to "moderate" for all devices. Equivalent test-retest results were observed for all raters using the Rolimeter, KLT and KT-1000, whereas measurement consistency with KiRA was given in the advanced examiners group only. Intra-rater ICC values (Range: SSD, ATT) were graded as "poor" to "moderate" for SSD values and "moderate" to "good" for ATT. SEMs were lowest for the Rolimeter and highest for KiRA. FPM were never obtained with the Rolimeter (0%), twice (2.1%) with the KT-1000, three times (3.1%) with the KLT and 33 times (34.4%) using KiRA. CONCLUSION: There is acceptable intra-rater but poor inter-rater reliability with all tested arthrometers. Measures of knee laxity are comparable between Rolimeter, KLT and KT-1000 but higher for KiRA. Clinically, the present study shows that repeated arthrometry measurements should always be performed by the same investigators.

Testing equivalence of survival before but not after end of follow-up.

For equivalence trials with survival outcomes, a popular testing approach is the elegant test for equivalence of two survival functions suggested by Wellek (Biometrics 49: 877-881, 1993). This test evaluates whether or not the difference between the true survival curves is practically irrelevant by specifying an equivalence margin on the hazard ratio under the proportional hazards assumption. However, this approach is based on extrapolating the behavior of the survival curves to the whole time axis, whereas in practice survival times are only observed until the end of follow-up. We propose a modification of Welleks test that only addresses equivalence until end of follow-up and derive the large sample properties of this test. Another issue is the proportional hazards assumption which may not be realistic. If this assumption is violated, one may severely misjudge the actual treatment effect with a hazard ratio quantification and wrongly declare equivalence. We suggest a non-parametric test for assessing survival equivalence within the follow-up period. We derive the large sample properties of this test and provide an approximation to the limiting distribution under some mild assumptions on the functional form of the difference between the two survival curves. Both suggestions are investigated by simulation and applied to a clinical trial on survival of gastric cancer patients.

Equivalence of regression curves sharing common parameters.

In clinical trials, the comparison of two different populations is a common problem. Nonlinear (parametric) regression models are commonly used to describe the relationship between covariates, such as concentration or dose, and a response variable in the two groups. In some situations, it is reasonable to assume some model parameters to be the same, for instance, the placebo effect or the maximum treatment effect. In this paper, we develop a (parametric) bootstrap test to establish the similarity of two regression curves sharing some common parameters. We show by theoretical arguments and by means of a simulation study that the new test controls its significance level and achieves a reasonable power. Moreover, it is demonstrated that under the assumption of common parameters, a considerably more powerful test can be constructed compared with the test that does not use this assumption. Finally, we illustrate the potential applications of the new methodology by a clinical trial example.

Using differential ratings of perceived exertion to assess agreement between coach and player perceptions of soccer training intensity: An exploratory investigation.

We aimed to assess the coach-player agreement of subjective soccer training loads via differential ratings of perceived exertion (dRPE). The coach initially underwent quantifiable familiarisation (blackness test) with the Borg CR100 scale. Data were collected from 16 semi-professional soccer players across seven consecutive training sessions. For the measurement of subjective training load, the coach and players provided dRPE (CR100) for legs (RPE-L), breathlessness (RPE-B) and technical exertion (RPE-T). Coach-prescribed dRPE were recorded prior to training, with coach observed and player reported dRPE collected post training. Statistical equivalence bounds for agreement between coach (prescribed and observed) and player reported dRPE scores were 4 arbitrary units on the CR100 and we used a probability outcome of likely (≥75%) to infer realistic equivalence. Following three familiarisation sessions, the coach improved their blackness test score from 39% to 83%. Coach observed and player reported RPE-T scores were likely equivalent, with all other comparisons not realistically equivalent. Since training prescription is coach-led, our data highlight the importance of accurate internal load measurement and feedback in soccer. The improved accuracy and precision of coach intensity estimation after three attempts at the blackness test suggests that this method could be worthwhile to researchers and practitioners employing dRPE.

Regulatory assessment of drug dissolution profiles comparability via maximum deviation.

In drug development, comparability of dissolution profiles of 2 different formulations is usually assessed using the similarity factor f2 . In practice, the drug dissolution profiles are deemed similar if the f2 exceeds 50, which occurs when a 10% maximum difference in the mean percentage of the dissolved drug at each time point between test and reference formulation is obtained. According to the Guideline on the Investigation of Bioequivalence (CPMP/EWP/QWP/1401/98 Rev. 1/ Corr **) use of the f2 is however restricted by a set of validity conditions. If some of these conditions are not satisfied, the f2 is not considered suitable, and alternative statistical methods are needed. In this article, we propose an inferential framework based on the maximum deviation between curves to test the comparability of drug dissolution profiles. The new methodology is applicable regardless whether the validity criteria of the f2 are met or not. Contrary to the f2 , this approach also integrates the variability of the measurements over time and not only their average. To benchmark our method, we performed simulations informed by 3 real case studies provided by the European Medicines Agency and extracted from dossiers submitted to the Centralised Procedure for Marketing Authorisation Application. In the scenarios of the simulation study, the new method controlled its type I error rate when the maximum deviation was greater than the similarity acceptance limit of 10%. The power exceeded 80% for small values of the maximum deviation, while the test was more conservative for intermediate ones. Our results were also very robust to sampling variations. Based on these positive findings, we encourage applicants to consider the new maximum deviation-based method as a valid alternative to the f2 , especially when the validity criteria of the latter are not met.