Nonlinear Regression Modelling: A Primer with Applications and Caveats.

Use of nonlinear statistical methods and models are ubiquitous in scientific research. However, these methods may not be fully understood, and as demonstrated here, commonly-reported parameter p-values and confidence intervals may be inaccurate. The gentle introduction to nonlinear regression modelling and comprehensive illustrations given here provides applied researchers with the needed overview and tools to appreciate the nuances and breadth of these important methods. Since these methods build upon topics covered in first and second courses in applied statistics and predictive modelling, the target audience includes practitioners and students alike. To guide practitioners, we summarize, illustrate, develop, and extend nonlinear modelling methods, and underscore caveats of Wald statistics using basic illustrations and give key reasons for preferring likelihood methods. Parameter profiling in multiparameter models and exact or near-exact versus approximate likelihood methods are discussed and curvature measures are connected with the failure of the Wald approximations regularly used in statistical software. The discussion in the main paper has been kept at an introductory level and it can be covered on a first reading; additional details given in the Appendices can be worked through upon further study. The associated online Supplementary Information also provides the data and R computer code which can be easily adapted to aid researchers to fit nonlinear models to their data.

Meta-analysis prediction intervals are under reported in sport and exercise medicine.

AIM: Prediction intervals are a useful measure of uncertainty for meta-analyses that capture the likely effect size of a new (similar) study based on the included studies. In comparison, confidence intervals reflect the uncertainty around the point estimate but provide an incomplete summary of the underlying heterogeneity in the meta-analysis. This study aimed to estimate (i) the proportion of meta-analysis studies that report a prediction interval in sports medicine; and (ii) the proportion of studies with a discrepancy between the reported confidence interval and a calculated prediction interval. METHODS: We screened, at random, 1500 meta-analysis studies published between 2012 and 2022 in highly ranked sports medicine and medical journals. Articles that used a random effect meta-analysis model were included in the study. We randomly selected one meta-analysis from each article to extract data from, which included the number of estimates, the pooled effect, and the confidence and prediction interval. RESULTS: Of the 1500 articles screened, 866 (514 from sports medicine) used a random effect model. The probability of a prediction interval being reported in sports medicine was 1.7% (95% CI = 0.9%, 3.3%). In medicine the probability was 3.9% (95% CI = 2.4%, 6.6%). A prediction interval was able to be calculated for 220 sports medicine studies. For 60% of these studies, there was a discrepancy in study findings between the reported confidence interval and the calculated prediction interval. Prediction intervals were 3.4 times wider than confidence intervals. CONCLUSION: Very few meta-analyses report prediction intervals and hence are prone to missing the impact of between-study heterogeneity on the overall conclusions. The widespread misinterpretation of random effect meta-analyses could mean that potentially harmful treatments, or those lacking a sufficient evidence base, are being used in practice. Authors, reviewers, and editors should be aware of the importance of prediction intervals.

Estimation of median survival time and its 95% confidence interval using SAS PROC LIFETEST.

Estimation of median survival and its 95% confidence interval depends on the choice of the survival function, standard error, and a method for constructing the confidence interval. This paper outlines several available possibilities in SAS® (version 9.4) PROC LIFETEST and compares them on theoretical grounds and using simulated data, with criteria: ability to estimate the 95% CI, coverage probability, interval width, and appropriateness for practical use. Data are generated with varying hazard patterns, N, % censoring, and censoring patterns (early, uniform, late, last visit). LIFETEST was run using the Kaplan-Meier and Nelson-Aalen estimators and the transformations available (linear, log, logit, complementary log-log, and arcsine square root). Using the Kaplan-Meier estimator with the logarithmic transformation as well as with the logit leads to a high frequency of LIFETEST not being able to estimate the 95% CI. The combination of Kaplan-Meier with the linear transformation is associated with poor coverage achieved. For small samples, late/last visit censoring has a negative effect on being able to estimate the 95% CI. Heavy early censoring can lead to low coverage of the 95% CI of median survival for sample sizes up to and including N = 40. The two combinations that are optimal for being able to estimate the 95% CI and having adequate coverage are the Kaplan-Meier estimator with complementary log-log transformation, and the Nelson-Aalen estimator with linear transformation. The former fares best on the third criterion (smaller width) and is also the SAS® default and validates the choice of default.

In vitro dissolution profile comparison using bootstrap bias corrected similarity factor, f2.

In vitro dissolution profile has been shown to be correlated with the drug absorption and has often been considered as a metric for assessing in vitro bioequivalence between a test product and corresponding reference one. Various methods have been developed to assess the similarity between two dissolution profiles. In particular, similarity factor f2 has been reviewed and discussed extensively in many statistical articles. Although the f2 lacks inferential statistical properties, the estimation of f2 and its various modified versions were the most widely used metric for comparing dissolution profiles. In this paper, we investigated performances of the naive f2 estimate method, bootstrap f2 confidence interval method and bias corrected-accelerated (BCa) bootstrap f2 confidence interval method for comparing dissolution profiles. Our studies show that naive f2 estimate method and BCa bootstrap f2 confidence interval method are unable to control the type I error rate. The bootstrap f2 confidence interval method can control the type I error rate under a specific level. However, it will cause great conservatism on the power of the test. To solve the potential issues of the previous methods, we recommended a bootstrap bias corrected (BC) f2 confidence interval method in this paper. The type I error rate, power and sensitivity among different f2 methods were compared based on simulations. The recommended bootstrap BC f2 confidence interval method shows better control of type I error than the naive f2 estimate method and BCa bootstrap f2 confidence interval method. It also provides better power than the bootstrap f2 confidence interval method.

Interval estimation of common risk difference for stratified unilateral and bilateral data✩.

In clinical trials, unilateral or bilateral data can usually be encountered if a subject contributes one or both of paired organs. For the bilateral data, responses from two paired body parts are correlated. In this paper, we study various confidence intervals of common risk difference in stratified unilateral and bilateral data based on the Dallal's model. Simulation results show that the score method outperforms other methods and provides coverage probability close to the nominal level and satisfactory coverage width. Hence, the method is recommended. In addition, the inverse hyperbolic tangent Wald-type become as optimal as the score method with the increase of sample sizes. An otolaryngology example is used to demonstrate the proposed methods.

Conditional bias adjusted estimator of treatment effect in 2-in-1 adaptive design.

For implementation of adaptive design, the adjustment of bias in treatment effect estimation becomes an increasingly important topic in recent years. While adaptive design literature traditionally focuses on the control of type I error rate and the adjustment of overall unconditional bias, the research on adjusting conditional bias has been limited. This paper proposes a conditional bias adjustment estimator of treatment effect under the context of 2-in-1 adaptive design and aims to provide a comprehensive investigation on their statistical properties including bias, mean squared error and coverage probability of confidence intervals. It demonstrated that conditional bias adjusted estimators greatly reduce the conditional bias and have similarly negligible unconditional bias compared with mean and median (unconditional) unbiased estimators. In addition, the test statistics is constructed based on the conditional bias adjustment estimators and compared with the naive unadjusted test.

Interval estimation of relative risks for combined unilateral and bilateral correlated data.

Measurements are generally collected as unilateral or bilateral data in clinical trials, epidemiology, or observational studies. For example, in ophthalmology studies, the primary outcome is often obtained from one eye or both eyes of an individual. In medical studies, the relative risk is usually the parameter of interest and is commonly used. In this article, we develop three confidence intervals for the relative risk for combined unilateral and bilateral correlated data under the equal dependence assumption. The proposed confidence intervals are based on maximum likelihood estimates of parameters derived using the Fisher scoring method. Simulation studies are conducted to evaluate the performance of proposed confidence intervals with respect to the empirical coverage probability, the mean interval width, and the ratio of mesial non-coverage probability to the distal non-coverage probability. We also compare the proposed methods with the confidence interval based on the method of variance estimates recovery and the confidence interval obtained from the modified Poisson regression model with correlated binary data. We recommend the score confidence interval for general applications because it best controls converge probabilities at the 95% level with reasonable mean interval width. We illustrate the methods with a real-world example.

A comparison of Bayesian and score methods for interval estimates of positive/negative likelihood ratios in support of diagnostic device performance evaluation.

BACKGROUND: Positive and negative likelihood ratios (PLR and NLR) are important metrics of accuracy for diagnostic devices with a binary output. However, the properties of Bayesian and frequentist interval estimators of PLR/NLR have not been extensively studied and compared. In this study, we explore the potential use of the Bayesian method for interval estimation of PLR/NLR, and, more broadly, for interval estimation of the ratio of two independent proportions. METHODS: We develop a Bayesian-based approach for interval estimation of PLR/NLR for use as a part of a diagnostic device performance evaluation. Our approach is applicable to a broader setting for interval estimation of any ratio of two independent proportions. We compare score and Bayesian interval estimators for the ratio of two proportions in terms of the coverage probability (CP) and expected interval width (EW) via extensive experiments and applications to two case studies. A supplementary experiment was also conducted to assess the performance of the proposed exact Bayesian method under different priors. RESULTS: Our experimental results show that the overall mean CP for Bayesian interval estimation is consistent with that for the score method (0.950 vs. 0.952), and the overall mean EW for Bayesian is shorter than that for score method (15.929 vs. 19.724). Application to two case studies showed that the intervals estimated using the Bayesian and frequentist approaches are very similar. DISCUSSION: Our numerical results indicate that the proposed Bayesian approach has a comparable CP performance with the score method while yielding higher precision (i.e. a shorter EW).

Determinants of intended prevention behaviour against mosquitoes and mosquito-borne viruses in the Netherlands and Spain using the MosquitoWise survey: cross-sectional study.

BACKGROUND: Recently, Europe has seen an emergence of mosquito-borne viruses (MBVs). Understanding citizens' perceptions of and behaviours towards mosquitoes and MBVs is crucial to reduce disease risk. We investigated and compared perceptions, knowledge, and determinants of citizens' behavioural intentions related to mosquitoes and MBVs in the Netherlands and Spain, to help improve public health interventions. METHODS: Using the validated MosquitoWise survey, data was collected through participant panels in Spain (N = 475) and the Netherlands (N = 438). Health Belief Model scores measuring behavioural intent, knowledge, and information scores were calculated. Confidence Interval-Based Estimation of Relevance was used, together with potential for change indexes, to identify promising determinants for improving prevention measure use. RESULTS: Spanish participants' responses showed slightly higher intent to use prevention measures compared to those of Dutch participants (29.1 and 28.2, respectively, p 0.03). Most participants in Spain (92.2%) and the Netherlands (91.8%) indicated they used at least one prevention measure, but differences were observed in which types they used. More Spanish participants indicated to have received information on mosquitoes and MBVs compared to Dutch participants. Spanish participants preferred health professional information sources, while Dutch participants favoured government websites. Determinants for intent to use prevention measures included "Knowledge", "Reminders to Use Prevention Measures", and "Information" in the Netherlands and Spain. Determinants for repellent use included "Perceived Benefits" and "Cues to Action", with "Perceived Benefits" having a high potential for behavioural change in both countries. "Self-Efficacy" and "Knowledge" were determinants in both countries for breeding site removal. CONCLUSION: This study found differences in knowledge between the Netherlands and Spain but similarities in determinants for intent to use prevention measures, intent to use repellents and intent to remove mosquito breeding sites. Identified determinants can be the focus for future public health interventions to reduce MBV risks.

A Confidence Interval for the Difference Between Standardized Regression Coefficients.

Researchers are often interested in comparing predictors, a practice commonly done via informal comparisons of standardized regression slopes. However, formal interval-based approaches offer advantages over informal comparison. Specifically, this article examines a delta-method-based confidence interval for the difference between two standardized regression coefficients, building upon previous work on confidence intervals for single coefficients. Using Monte Carlo simulation studies, the proposed approach is evaluated at finite sample sizes with respect to coverage rate, interval width, Type I error rate, and statistical power under a variety of conditions, and is shown to outperform an alternative approach that uses the standard covariance matrix found in regression textbooks. Additional simulations evaluate current software implementations, small sample performance, and multiple comparison procedures for simultaneously testing multiple differences of interest. Guidance on sample size planning for narrow confidence intervals, an R function to conduct the proposed method, and two empirical demonstrations are provided. The goal is to offer researchers a different tool in their toolbox for when comparisons among standardized coefficients are desired, as a supplement to, rather than a replacement for, other potentially useful analyses.

Analysis of two binomial proportions in noninferiority confirmatory trials.

In this article, we propose considering an approximate exact score (AES) test for noninferiority comparisons and we derive its test-based confidence interval for the difference between two independent binomial proportions. This test was published in the literature, but not its associated confidence interval. The p-value for this test is obtained by using exact binomial probabilities with the nuisance parameter being replaced by its restricted maximum likelihood estimate. Calculated type I errors revealed that the AES method has important advantages for noninferiority comparisons over popular asymptotic methods for adequately powered confirmatory clinical trials, at 80% or 90% statistical power. For unbalanced sample sizes of the compared groups, type I errors for the asymptotic score method were shown to be higher than the nominal level in a systematic pattern over a range of true proportions, but the AES method did not suffer from such a problem. On average, the true type I error of the AES method was closer to the nominal level than all considered methods in the empirical comparisons. In rare cases, type I errors of the AES test exceeded the nominal level, but only by a small amount. Presented examples showed that the AES method can be more attractive in practice than practical exact methods. In addition, p-value and confidence interval of the AES method can be obtained in <30 s of computer time for most confirmatory trials. Theoretical arguments, combined with empirical evidence and fast computation time should make the AES method attractive in statistical practice.

Optimal Cut-Point Selection Methods Under Binary Classification When Subclasses Are Involved.

In practice, we often encounter binary classification problems where both main classes consist of multiple subclasses. For example, in an ovarian cancer study where biomarkers were evaluated for their accuracy of distinguishing noncancer cases from cancer cases, the noncancer class consists of healthy subjects and benign cases, while the cancer class consists of subjects at both early and late stages. This article aims to provide a large number of optimal cut-point selection methods for such setting. Furthermore, we also study confidence interval estimation of the optimal cut-points. Simulation studies are carried out to explore the performance of the proposed cut-point selection methods as well as confidence interval estimation methods. A real ovarian cancer data set is analyzed using the proposed methods.

Generalized Confidence Intervals for Ratios of Standard Deviations Based on Log-Normal Distribution when Times Follow Weibull Distributions.

Modern anesthetic drugs ensure the efficacy of general anesthesia. Goals include reducing variability in surgical, tracheal extubation, post-anesthesia care unit, or intraoperative response recovery times. Generalized confidence intervals based on the log-normal distribution compare variability between groups, specifically ratios of standard deviations. The alternative statistical approaches, performing robust variance comparison tests, give P-values, not point estimates nor confidence intervals for the ratios of the standard deviations. We performed Monte-Carlo simulations to learn what happens to confidence intervals for ratios of standard deviations of anesthesia-associated times when analyses are based on the log-normal, but the true distributions are Weibull. We used simulation conditions comparable to meta-analyses of most randomized trials in anesthesia, n ≈ 25 and coefficients of variation ≈ 0.30 . The estimates of the ratios of standard deviations were positively biased, but slightly, the ratios being 0.11% to 0.33% greater than nominal. In contrast, the 95% confidence intervals were very wide (i.e., > 95% of P ≥ 0.05). Although substantive inferentially, the differences in the confidence limits were small from a clinical or managerial perspective, with a maximum absolute difference in ratios of 0.016. Thus, P < 0.05 is reliable, but investigators should plan for Type II errors at greater than nominal rates.

Empirical methods for the validation of time-to-event mathematical models taking into account uncertainty and variability: application to EGFR + lung adenocarcinoma.

BACKGROUND: Over the past several decades, metrics have been defined to assess the quality of various types of models and to compare their performance depending on their capacity to explain the variance found in real-life data. However, available validation methods are mostly designed for statistical regressions rather than for mechanistic models. To our knowledge, in the latter case, there are no consensus standards, for instance for the validation of predictions against real-world data given the variability and uncertainty of the data. In this work, we focus on the prediction of time-to-event curves using as an application example a mechanistic model of non-small cell lung cancer. We designed four empirical methods to assess both model performance and reliability of predictions: two methods based on bootstrapped versions of parametric statistical tests: log-rank and combined weighted log-ranks (MaxCombo); and two methods based on bootstrapped prediction intervals, referred to here as raw coverage and the juncture metric. We also introduced the notion of observation time uncertainty to take into consideration the real life delay between the moment when an event happens, and the moment when it is observed and reported. RESULTS: We highlight the advantages and disadvantages of these methods according to their application context. We have shown that the context of use of the model has an impact on the model validation process. Thanks to the use of several validation metrics we have highlighted the limit of the model to predict the evolution of the disease in the whole population of mutations at the same time, and that it was more efficient with specific predictions in the target mutation populations. The choice and use of a single metric could have led to an erroneous validation of the model and its context of use. CONCLUSIONS: With this work, we stress the importance of making judicious choices for a metric, and how using a combination of metrics could be more relevant, with the objective of validating a given model and its predictions within a specific context of use. We also show how the reliability of the results depends both on the metric and on the statistical comparisons, and that the conditions of application and the type of available information need to be taken into account to choose the best validation strategy.

Propagating uncertainty about molecular evolution models and prior distributions to phylogenetic trees.

Phylogenetic trees constructed from molecular sequence data rely on largely arbitrary assumptions about the substitution model, the distribution of substitution rates across sites, the version of the molecular clock, and, in the case of Bayesian inference, the prior distribution. Those assumptions affect results reported in the form of clade probabilities and error bars on divergence times and substitution rates. Overlooking the uncertainty in the assumptions leads to overly confident conclusions in the form of inflated clade probabilities and short confidence intervals or credible intervals. This paper demonstrates how to propagate that uncertainty by combining the models considered along with all of their assumptions, including their prior distributions. The combined models incorporate much more of the uncertainty than Bayesian model averages since the latter tend to settle on a single model due to the higher-level assumption that one of the models is true. Nucleotide sequence data illustrates the proposed model combination method.

Methods for two nonstandard problems arising from the Luria-Delbrück experiment.

The fluctuation experiment, devised by Luria and Delbrück in 1943, remains the method of choice for measuring microbial mutation rates in the laboratory. While most inference problems commonly encountered in a fluctuation experiment can be tackled by existing standard algorithms, investigators from time to time run into nonstandard problems not amenable to any existing algorithms. A major obstacle to solving these nonstandard problems is the construction of confidence intervals for mutation rates. This note describes methods for two important categories of nonstandard problems, namely, pooling data from separate experiments and analyzing grouped mutant count data, focusing on the construction of likelihood ratio confidence intervals. In addition to illustrative examples using real-world data, simulation results are presented to help assess the proposed methods.

Assessing the robustness of results from clinical trials and meta-analyses with the fragility index.

The fragility index has been increasingly used to assess the robustness of the results of clinical trials since 2014. It aims at finding the smallest number of event changes that could alter originally statistically significant results. Despite its popularity, some researchers have expressed several concerns about the validity and usefulness of the fragility index. It offers a comprehensive review of the fragility index's rationale, calculation, software, and interpretation, with emphasis on application to studies in obstetrics and gynecology. This article presents the fragility index in the settings of individual clinical trials, standard pairwise meta-analyses, and network meta-analyses. Moreover, this article provides worked examples to demonstrate how the fragility index can be appropriately calculated and interpreted. In addition, the limitations of the traditional fragility index and some solutions proposed in the literature to address these limitations were reviewed. In summary, the fragility index is recommended to be used as a supplemental measure in the reporting of clinical trials and a tool to communicate the robustness of trial results to clinicians. Other considerations that can aid in the fragility index's interpretation include the loss to follow-up and the likelihood of data modifications that achieve the loss of statistical significance.

Exact-corrected confidence interval for risk difference in noninferiority binomial trials.

A novel confidence interval estimator is proposed for the risk difference in noninferiority binomial trials. The proposed confidence interval, which is dependent on the prespecified noninferiority margin, is consistent with an exact unconditional test that preserves the type-I error and has improved power, particularly for smaller sample sizes, compared to the confidence interval by Chan and Zhang. The improved performance of the proposed confidence interval is theoretically justified and demonstrated with simulations and examples. An R package is also distributed that implements the proposed methods along with other confidence interval estimators.

A seasonality-adjusted sequential test for vaccine safety surveillance.

Post-market active safety monitoring is important for the timely capture of safety signals associated with exposure to a new vaccine or drug. The group sequential analysis is a common method employed in safety surveillance. Specifically, it compares the post-vaccination incidence of adverse event (AE) in a vaccinated population with a pre-specified reference level by sequentially conducting hypothesis testing during the surveillance. When the number of AEs is "too high", a safety signal is identified. If the null hypothesis is never rejected, the vaccine is considered safe. Such an approach does not account for either the variation in determining the reference risk from a control population or the seasonality effect. Furthermore, not rejecting the null could be due to a lack of power and cannot always be interpreted as proof of safety. In this paper, we proposed a new group sequential test procedure fully accounting for both seasonality and variation from the historical controls. More importantly, we proposed to construct a confidence interval for the relative AE risk between the exposed and control groups at the end of the study, which can be used to quantify the safety of the vaccine. The proposed method is illustrated via real-data examples on anaphylaxis and examined by extensive simulation studies.

Clustering of trajectories with mixed effects classification model: Inference taking into account classification uncertainties.

Classifying patient biomarker trajectories into groups has become frequent in clinical research. Mixed effects classification models can be used to model the heterogeneity of longitudinal data. The estimated parameters of typical trajectories and the partition can be provided by the classification version of the expectation maximization algorithm, named CEM. However, the variance of the parameter estimates obtained underestimates the true variance because classification uncertainties are not taken into account. This article takes into account these uncertainties by using the stochastic EM algorithm (SEM), a stochastic version of the CEM algorithm, after convergence of the CEM algorithm. The simulations showed correct coverage probabilities of the 95% confidence intervals (close to 95% except for scenarios with high bias in typical trajectories). The method was applied on a trial, called low-cyclo, that compared the effects of low vs standard cyclosporine A doses on creatinine levels after cardiac transplantation. It identified groups of patients for whom low-dose cyclosporine may be relevant, but with high uncertainty on the dose-effect estimate.