Sample size determination for interval estimation of the prevalence of a sensitive attribute under non-randomized response models.

A sufficient number of participants should be included to adequately address the research interest in the surveys with sensitive questions. In this paper, sample size formulas/iterative algorithms are developed from the perspective of controlling the confidence interval width of the prevalence of a sensitive attribute under four non-randomized response models: the crosswise model, parallel model, Poisson item count technique model and negative binomial item count technique model. In contrast to the conventional approach for sample size determination, our sample size formulas/algorithms explicitly incorporate an assurance probability of controlling the width of a confidence interval within the pre-specified range. The performance of the proposed methods is evaluated with respect to the empirical coverage probability, empirical assurance probability and confidence width. Simulation results show that all formulas/algorithms are effective and hence are recommended for practical applications. A real example is used to illustrate the proposed methods.

Multiple test procedures of disease prevalence based on stratified partially validated series in the presence of a gold standard.

This paper discusses the problem of disease prevalence in clinical studies, focusing on multiple comparisons based on stratified partially validated series in the presence of a gold standard. Five test statistics, including two Wald-type test statistics, the inverse hyperbolic tangent transformation test statistic, likelihood ratio test statistic, and score test statistic, are proposed to conduct multiple comparisons. To control the overall type I error rate, several adjustment procedures are developed, namely the Bonferroni, Single-step adjusted MaxT, Single-step adjusted MinP, Holm's Step-down, and Hochberg's step-up procedures, based on these test statistics. The performance of the proposed methods is evaluated through simulation studies in terms of the empirical type I error rate and empirical power. Simulation results show that the Single-step adjusted MaxT procedure and Single-step adjusted MinP procedure generally outperform the other three procedures, and these two test procedures based on all test statistics have satisfactory performance. Notably, the Single-step adjusted MinP procedure tends to exhibit higher empirical power than the Single-step adjusted MaxT procedure. Furthermore, the Step-down and Step-up procedures show greater power compared to the Bonferroni method. The study also observes that as the validated ratio increases, the empirical type I errors of all test procedures approach the nominal level while maintaining higher power. Two real examples are presented to illustrate the proposed methods.

A rank-based approach to design and analysis of pretest-posttest randomized trials, with application to COVID-19 ordinal scale data.

Randomized controlled trials with a pretest-posttest design frequently yield ordered categorical outcome data. Focusing on the estimation of the win probability that a treated participant would have a better score than (or win over) a control participant, we developed methods for analysis and sample size planning for such trials. We exploited the analysis of covariance framework with the dependent variable being individual participants' win fractions at posttest and the covariate being the win fractions at pretest. The win fractions were obtained using the mid-ranks of the ordinal data. Simulation evaluation based on a recent randomized trial on COVID-19 suggests that the methods perform very well. A sample SAS code for data analysis is presented.

Parametric and nonparametric methods for confidence intervals and sample size planning for win probability in parallel-group randomized trials with Likert item and Likert scale data.

Data on the Likert scale are ubiquitous in medical research, including randomized trials. Statistical analysis of such data may be conducted using the means of raw scores or the rank information of the scores. In the context of parallel-group randomized trials, we quantify treatment effects by the probability that a subject in the treatment group has a better score than (or a win over) a subject in the control group. Asymptotic parametric and nonparametric confidence intervals for this win probability and associated sample size formulas are derived for studies with only follow-up scores, and those with both baseline and follow-up measurements. We assessed the performance of both the parametric and nonparametric approaches using simulation studies based on real studies with Likert item and Likert scale data. The simulation results demonstrate that even without baseline adjustment, the parametric methods did not perform well, in terms of bias, interval coverage percentage, balance of tail error, and assurance of achieving a pre-specified precision. In contrast, the nonparametric approach performed very well for both the unadjusted and adjusted win probability. We illustrate the methods with two examples: one using Likert item data and the other using Like scale data. We conclude that non-parametric methods are preferable for two-group randomization trials with Likert data. Illustrative SAS code for the nonparametric approach using existing procedures is provided.

Confidence intervals for assessing equivalence of two treatments with combined unilateral and bilateral data.

Responses from the paired organs are generally highly correlated in bilateral studies, statistical procedures ignoring the correlation could lead to incorrect results. Note the intraclass correlation in the study of combined unilateral and bilateral outcomes; 11 confidence intervals (CIs) including 7 asymptotic CIs and 4 Bootstrap-resampling CIs for assessing the equivalence of 2 treatments are derived under Rosner's correlated binary data model. Performance is evaluated with respect to the empirical coverage probability (ECP), the empirical coverage width (ECW) and the ratio of the mesial non-coverage probability to the non-coverage probability (RMNCP) via simulation studies. Simulation results show that (i) all CIs except for the Wald CI and the bias-corrected Bootstrap percentile CI generally produce satisfactory ECPs and hence are recommended; (ii) all CIs except for the bias-corrected Bootstrap percentile CI provide preferred RMNCPs and are more symmetrical; (iii) as the measurement of the dependence increases, the ECWs of all CIs except for the score CI and the profile likelihood CI show increasing patterns that look like linear, while there is no obvious pattern on the ECPs of all CIs except for the profile likelihood CI. A data set from an otolaryngologic study is used to illustrate the proposed methods.

Confidence interval construction for proportion difference from partially validated series with two fallible classifiers.

This article investigates the confidence interval (CI) construction of proportion difference for two independent partially validated series under the double-sampling scheme in which both classifiers are fallible. Several CIs based on the variance estimates recovery method of combining confidence limits from asymptotic, bootstrap, and Bayesian methods for two independent binomial proportions are developed under two models. Simulation results show that all CIs except for the bootstrap percentile-t CI and Bayesian credible interval with uniform prior under the independence model and all CIs under the dependence model generally perform well and are recommended. Two examples are used to illustrate the methodologies.

Sample Size Determination for Interval Estimation of the Prevalence of a Sensitive Attribute Under Randomized Response Models.

Studies with sensitive questions should include a sufficient number of respondents to adequately address the research interest. While studies with an inadequate number of respondents may not yield significant conclusions, studies with an excess of respondents become wasteful of investigators' budget. Therefore, it is an important step in survey sampling to determine the required number of participants. In this article, we derive sample size formulas based on confidence interval estimation of prevalence for four randomized response models, namely, the Warner's randomized response model, unrelated question model, item count technique model and cheater detection model. Specifically, our sample size formulas control, with a given assurance probability, the width of a confidence interval within the planned range. Simulation results demonstrate that all formulas are accurate in terms of empirical coverage probabilities and empirical assurance probabilities. All formulas are illustrated using a real-life application about the use of unethical tactics in negotiation.

Homogeneity testing for binomial proportions under stratified double-sampling scheme with two fallible classifiers.

This article investigates the homogeneity testing problem of binomial proportions for stratified partially validated data obtained by double-sampling method with two fallible classifiers. Several test procedures, including the weighted-least-squares test with/without log-transformation, logit-transformation and double log-transformation, and likelihood ratio test and score test, are developed to test the homogeneity under two models, distinguished by conditional independence assumption of two classifiers. Simulation results show that score test performs better than other tests in the sense that the empirical size is generally controlled around the nominal level, and hence be recommended to practical applications. Other tests also perform well when both binomial proportions and sample sizes are not small. Approximate sample sizes based on score test, likelihood ratio test and the weighted-least-squares test with double log-transformation are generally accurate in terms of the empirical power and type I error rate with the estimated sample sizes, and hence be recommended. An example from the malaria study is illustrated by the proposed methodologies.

Comparison of disease prevalence in two populations under double-sampling scheme with two fallible classifiers.

A disease prevalence can be estimated by classifying subjects according to whether they have the disease. When gold-standard tests are too expensive to be applied to all subjects, partially validated data can be obtained by double-sampling in which all individuals are classified by a fallible classifier, and some of individuals are validated by the gold-standard classifier. However, it could happen in practice that such infallible classifier does not available. In this article, we consider two models in which both classifiers are fallible and propose four asymptotic test procedures for comparing disease prevalence in two groups. Corresponding sample size formulae and validated ratio given the total sample sizes are also derived and evaluated. Simulation results show that (i) Score test performs well and the corresponding sample size formula is also accurate in terms of the empirical power and size in two models; (ii) the Wald test based on the variance estimator with parameters estimated under the null hypothesis outperforms the others even under small sample sizes in Model II, and the sample size estimated by this test is also accurate; (iii) the estimated validated ratios based on all tests are accurate. The malarial data are used to illustrate the proposed methodologies.

Construction of confidence intervals for the risk differences in stratified design with correlated bilateral data.

A stratified study is often designed for adjusting a confounding effect or effect of different centers/groups in two treatments or diagnostic tests, and the risk difference is one of the most frequently used indices in comparing efficiency between two treatments or diagnostic tests. This article presented five simultaneous confidence intervals (CIs) for risk differences in stratified bilateral designs accounting for the intraclass correlation and developed seven CIs for the common risk difference under the homogeneity assumption. The performance of the CIs is evaluated with respect to the empirical coverage probabilities, empirical coverage widths and ratios of mesial noncoverage probability and the noncoverage probability under various scenarios. Empirical results show that Wald simultaneous CI, Haldane simultaneous CI, Score simultaneous CI based on Bonferroni method and simultaneous CI based on bootstrap-resampling method perform satisfactorily and hence be recommended for applications, the CI based on the weighted-least-square (WLS) estimator, the CIs based on Mantel-Haenszel estimator, the CI based on Cochran statistic and the CI based on Score statistic for the common risk difference behave well even under small sample sizes. A real data example is used to demonstrate the proposed methodologies.

Test procedure and sample size determination for a proportion study using a double-sampling scheme with two fallible classifiers.

Double sampling is usually applied to collect necessary information for situations in which an infallible classifier is available for validating a subset of the sample that has already been classified by a fallible classifier. Inference procedures have previously been developed based on the partially validated data obtained by the double-sampling process. However, it could happen in practice that such infallible classifier or gold standard does not exist. In this article, we consider the case in which both classifiers are fallible and propose asymptotic and approximate unconditional test procedures based on six test statistics for a population proportion and five approximate sample size formulas based on the recommended test procedures under two models. Our results suggest that both asymptotic and approximate unconditional procedures based on the score statistic perform satisfactorily for small to large sample sizes and are highly recommended. When sample size is moderate or large, asymptotic procedures based on the Wald statistic with the variance being estimated under the null hypothesis, likelihood rate statistic, log- and logit-transformation statistics based on both models generally perform well and are hence recommended. The approximate unconditional procedures based on the log-transformation statistic under Model I, Wald statistic with the variance being estimated under the null hypothesis, log- and logit-transformation statistics under Model II are recommended when sample size is small. In general, sample size formulae based on the Wald statistic with the variance being estimated under the null hypothesis, likelihood rate statistic and score statistic are recommended in practical applications. The applicability of the proposed methods is illustrated by a real-data example.

Interval estimation for a proportion using a double-sampling scheme with two fallible classifiers.

Double-sampling schemes using one classifier assessing the whole sample and another classifier assessing a subset of the sample have been introduced for reducing classification errors when an infallible or gold standard classifier is unavailable or impractical. Inference procedures have previously been proposed for situations where an infallible classifier is available for validating a subset of the sample that has already been classified by a fallible classifier. Here, we consider the case where both classifiers are fallible, proposing and evaluating several confidence interval procedures for a proportion under two models, distinguished by the assumption regarding ascertainment of two classifiers. Simulation results suggest that the modified Wald-based confidence interval, Score-based confidence interval, two Bayesian credible intervals, and the percentile Bootstrap confidence interval performed reasonably well even for small binomial proportions and small validated sample under the model with the conditional independent assumption, and the confidence interval derived from the Wald test with nuisance parameters appropriately evaluated, likelihood ratio-based confidence interval, Score-based confidence interval, and the percentile Bootstrap confidence interval performed satisfactory in terms of coverage under the model without the conditional independent assumption. Moreover, confidence intervals based on log- and logit-transformations also performed well when the binomial proportion and the ratio of the validated sample are not very small under two models. Two examples were used to illustrate the procedures.

Sample size determination for disease prevalence studies with partially validated data.

Disease prevalence is an important topic in medical research, and its study is based on data that are obtained by classifying subjects according to whether a disease has been contracted. Classification can be conducted with high-cost gold standard tests or low-cost screening tests, but the latter are subject to the misclassification of subjects. As a compromise between the two, many research studies use partially validated datasets in which all data points are classified by fallible tests, and some of the data points are validated in the sense that they are also classified by the completely accurate gold-standard test. In this article, we investigate the determination of sample sizes for disease prevalence studies with partially validated data. We use two approaches. The first is to find sample sizes that can achieve a pre-specified power of a statistical test at a chosen significance level, and the second is to find sample sizes that can control the width of a confidence interval with a pre-specified confidence level. Empirical studies have been conducted to demonstrate the performance of various testing procedures with the proposed sample sizes. The applicability of the proposed methods are illustrated by a real-data example.

Confidence intervals for proportion difference from two independent partially validated series.

Partially validated series are common when a gold-standard test is too expensive to be applied to all subjects, and hence a fallible device is used accordingly to measure the presence of a characteristic of interest. In this article, confidence interval construction for proportion difference between two independent partially validated series is studied. Ten confidence intervals based on the method of variance estimates recovery (MOVER) are proposed, with each using the confidence limits for the two independent binomial proportions obtained by the asymptotic, Logit-transformation, Agresti-Coull and Bayesian methods. The performances of the proposed confidence intervals and three likelihood-based intervals available in the literature are compared with respect to the empirical coverage probability, confidence width and ratio of mesial non-coverage to non-coverage probability. Our empirical results show that (1) all confidence intervals exhibit good performance in large samples; (2) confidence intervals based on MOVER combining the confidence limits for binomial proportions based on Wilson, Agresti-Coull, Logit-transformation, Bayesian (with three priors) methods perform satisfactorily from small to large samples, and hence can be recommended for practical applications. Two real data sets are analysed to illustrate the proposed methods.

Simultaneous confidence intervals of risk differences in stratified paired designs.

In stratified matched-pair studies, risk difference between two proportions is one of the most frequently used indices in comparing efficiency between two treatments or diagnostic tests. This article presents five simultaneous confidence intervals and two bootstrap simultaneous confidence intervals for risk differences in stratified matched-pair designs. The proposed confidence intervals are evaluated with respect to their coverage probabilities, expected widths, and ratios of the mesial noncoverage to noncoverage probability. Empirical results show that (1) hybrid simultaneous confidence intervals outperform nonhybrid simultaneous confidence intervals; (2) hybrid simultaneous confidence intervals based on median estimator outperform those based on maximum likelihood estimator; and (3) hybrid simultaneous confidence intervals incorporated with Wilson score and Agresti coull intervals and the bootstrap t-percentile simultaneous interval based on median unbiased estimators behave satisfactorily for small to large sample sizes in the sense that their empirical coverage probabilities are close to the prespecified nominal confidence level, and their ratios of the mesial noncoverage to noncoverage probabilities lie in [0.4,0.6] and are hence recommended. Real examples from clinical studies are used to illustrate the proposed methodologies.

Comparison of disease prevalence in two populations in the presence of misclassification.

Comparing disease prevalence in two groups is an important topic in medical research, and prevalence rates are obtained by classifying subjects according to whether they have the disease. Both high-cost infallible gold-standard classifiers or low-cost fallible classifiers can be used to classify subjects. However, statistical analysis that is based on data sets with misclassifications leads to biased results. As a compromise between the two classification approaches, partially validated sets are often used in which all individuals are classified by fallible classifiers, and some of the individuals are validated by the accurate gold-standard classifiers. In this article, we develop several reliable test procedures and approximate sample size formulas for disease prevalence studies based on the difference between two disease prevalence rates with two independent partially validated series. Empirical studies show that (i) the Score test produces close-to-nominal level and is preferred in practice; and (ii) the sample size formula based on the Score test is also fairly accurate in terms of the empirical power and type I error rate, and is hence recommended. A real example from an aplastic anemia study is used to illustrate the proposed methodologies.

Test procedures for disease prevalence with partially validated data.

Investigating the prevalence of a disease is an important topic in medical studies. Such investigations are usually based on the classification results of a group of subjects according to whether they have the disease. To classify subjects, screening tests that are inexpensive and nonintrusive to the test subjects are frequently used to produce results in a timely manner. However, such screening tests may suffer from high levels of misclassification. Although it is often possible to design a gold-standard test or device that is not subject to misclassification, such devices are usually costly and time-consuming, and in some cases intrusive to the test subjects. As a compromise between these two approaches, it is possible to use data that are obtained by the method of double-sampling. In this article, we derive and investigate four test statistics for testing a hypothesis on disease prevalence with double-sampling data. The test statistics are implemented through both the asymptotic method suitable for large samples and approximate unconditional method suitable for small samples. Our simulation results show that the approximate unconditional method usually produces a more satisfactory empirical type I error rate and power than its asymptotic counterpart, especially for small to moderate sample sizes. The results also suggest that the score test and the Wald test based on an estimate of variance with parameters estimated under the null hypothesis outperform the others. An real example is used to illustrate the proposed methods.

Asymptotic confidence interval construction for proportion difference in medical studies with bilateral data.

Bilateral dichotomous data are very common in modern medical comparative studies (e.g. comparison of two treatments in ophthalmologic, orthopaedic and otolaryngologic studies) in which information involving paired organs (e.g. eyes, ears and hips) is available from each subject. In this article, we study various confidence interval estimators for proportion difference based on Wald-type statistics, Fieller theorem, likelihood ratio statistic, score statistics and bootstrap resampling method under the dependence or/and independence models for bilateral binary data. Performance is evaluated with respect to the coverage probability and expected width via simulation studies. Our empirical results show that (1) ignoring the dependence feature of bilateral data could lead to severely incorrect coverage probabilities; and (2) Wald-type, score-type and bootstrap confidence intervals based on the dependence model perform satisfactorily for small to large sample sizes in the sense that their empirical coverage probabilities are close to the pre-specified nominal confidence level and are hence recommended. A real data from an otolaryngologic study is used to illustrate the proposed methods.

Sample size for testing difference between two proportions for the bilateral-sample design.

In this article, we consider approximate sample size formulas for testing difference between two proportions for bilateral studies with binary outcomes. Sample size formulas are derived to achieve a prespecified power of a statistical test at a prechosen significance level. Four statistical tests are considered. Simulation studies are conducted to investigate the accuracy of various formulas. In general, the sample size formula for Rosner's statistic based on the dependence assumption is highly recommended in the sense that its actual power is satisfactorily close to the desired power level. An example from an otolaryngological study is used to demonstrate the proposed methodologies.