Handling missing data in randomization tests for single-case experiments: A simulation study.

Single-case experiments have become increasingly popular in psychological and educational research. However, the analysis of single-case data is often complicated by the frequent occurrence of missing or incomplete data. If missingness or incompleteness cannot be avoided, it becomes important to know which strategies are optimal, because the presence of missing data or inadequate data handling strategies may lead to experiments no longer "meeting standards" set by, for example, the What Works Clearinghouse. For the examination and comparison of strategies to handle missing data, we simulated complete datasets for ABAB phase designs, randomized block designs, and multiple-baseline designs. We introduced different levels of missingness in the simulated datasets by randomly deleting 10%, 30%, and 50% of the data. We evaluated the type I error rate and statistical power of a randomization test for the null hypothesis that there was no treatment effect under these different levels of missingness, using different strategies for handling missing data: (1) randomizing a missing-data marker and calculating all reference statistics only for the available data points, (2) estimating the missing data points by single imputation using the state space representation of a time series model, and (3) multiple imputation based on regressing the available data points on preceding and succeeding data points. The results are conclusive for the conditions simulated: The randomized-marker method outperforms the other two methods in terms of statistical power in a randomization test, while keeping the type I error rate under control.

A randomization test wrapper for synthesizing single-case experiments using multilevel models: A Monte Carlo simulation study.

Multilevel models (MLMs) have been proposed in single-case research, to synthesize data from a group of cases in a multiple-baseline design (MBD). A limitation of this approach is that MLMs require several statistical assumptions that are often violated in single-case research. In this article we propose a solution to this limitation by presenting a randomization test (RT) wrapper for MLMs that offers a nonparametric way to evaluate treatment effects, without making distributional assumptions or an assumption of random sampling. We present the rationale underlying the proposed technique and validate its performance (with respect to Type I error rate and power) as compared to parametric statistical inference in MLMs, in the context of evaluating the average treatment effect across cases in an MBD. We performed a simulation study that manipulated the numbers of cases and of observations per case in a dataset, the data variability between cases, the distributional characteristics of the data, the level of autocorrelation, and the size of the treatment effect in the data. The results showed that the power of the RT wrapper is superior to the power of parametric tests based on F distributions for MBDs with fewer than five cases, and that the Type I error rate of the RT wrapper is controlled for bimodal data, whereas this is not the case for traditional MLMs.

Nonparametric meta-analysis for single-case research: Confidence intervals for combined effect sizes.

In this article we present a nonparametric technique for meta-analyzing randomized single-case experiments by using inverted randomization tests to calculate nonparametric confidence intervals for combined effect sizes (CICES). Over the years, several proposals for single-case meta-analysis have been made, but most of these proposals assume either specific population characteristics (e.g., heterogeneity of variances or normality) or independent observations. However, such assumptions are seldom plausible in single-case research. The CICES technique does not require such assumptions, but only assumes that the combined effect size of multiple randomized single-case experiments can be modeled as a constant difference in the phase means. CICES can be used to synthesize the results from various single-case alternation designs, single-case phase designs, or a combination of the two. Furthermore, the technique can be used with different standardized or unstandardized effect size measures. In this article, we explain the rationale behind the CICES technique and provide illustrations with empirical as well as hypothetical datasets. In addition, we discuss the strengths and weaknesses of this technique and offer some possibilities for future research. We have implemented the CICES technique for single-case meta-analysis in a freely available R function.

Randomized single-case AB phase designs: Prospects and pitfalls.

Single-case experimental designs (SCEDs) are increasingly used in fields such as clinical psychology and educational psychology for the evaluation of treatments and interventions in individual participants. The AB phase design, also known as the interrupted time series design, is one of the most basic SCEDs used in practice. Randomization can be included in this design by randomly determining the start point of the intervention. In this article, we first introduce this randomized AB phase design and review its advantages and disadvantages. Second, we present some data-analytical possibilities and pitfalls related to this design and show how the use of randomization tests can mitigate or remedy some of these pitfalls. Third, we demonstrate that the Type I error of randomization tests in randomized AB phase designs is under control in the presence of unexpected linear trends in the data. Fourth, we report the results of a simulation study investigating the effect of unexpected linear trends on the power of the randomization test in randomized AB phase designs. The implications of these results for the analysis of randomized AB phase designs are discussed. We conclude that randomized AB phase designs are experimentally valid, but that the power of these designs is sufficient only for large treatment effects and large sample sizes. For small treatment effects and small sample sizes, researchers should turn to more complex phase designs, such as randomized ABAB phase designs or randomized multiple-baseline designs.

The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study.

The conditional power (CP) of the randomization test (RT) was investigated in a simulation study in which three different single-case effect size (ES) measures were used as the test statistics: the mean difference (MD), the percentage of nonoverlapping data (PND), and the nonoverlap of all pairs (NAP). Furthermore, we studied the effect of the experimental design on the RT's CP for three different single-case designs with rapid treatment alternation: the completely randomized design (CRD), the randomized block design (RBD), and the restricted randomized alternation design (RRAD). As a third goal, we evaluated the CP of the RT for three types of simulated data: data generated from a standard normal distribution, data generated from a uniform distribution, and data generated from a first-order autoregressive Gaussian process. The results showed that the MD and NAP perform very similarly in terms of CP, whereas the PND performs substantially worse. Furthermore, the RRAD yielded marginally higher power in the RT, followed by the CRD and then the RBD. Finally, the power of the RT was almost unaffected by the type of the simulated data. On the basis of the results of the simulation study, we recommend at least 20 measurement occasions for single-case designs with a randomized treatment order that are to be evaluated with an RT using a 5% significance level. Furthermore, we do not recommend use of the PND, because of its low power in the RT.

Confidence intervals for single-case effect size measures based on randomization test inversion.

In the current paper, we present a method to construct nonparametric confidence intervals (CIs) for single-case effect size measures in the context of various single-case designs. We use the relationship between a two-sided statistical hypothesis test at significance level α and a 100 (1 - α) % two-sided CI to construct CIs for any effect size measure θ that contain all point null hypothesis θ values that cannot be rejected by the hypothesis test at significance level α. This method of hypothesis test inversion (HTI) can be employed using a randomization test as the statistical hypothesis test in order to construct a nonparametric CI for θ. We will refer to this procedure as randomization test inversion (RTI). We illustrate RTI in a situation in which θ is the unstandardized and the standardized difference in means between two treatments in a completely randomized single-case design. Additionally, we demonstrate how RTI can be extended to other types of single-case designs. Finally, we discuss a few challenges for RTI as well as possibilities when using the method with other effect size measures, such as rank-based nonoverlap indices. Supplementary to this paper, we provide easy-to-use R code, which allows the user to construct nonparametric CIs according to the proposed method.

Consistency in Single-Case ABAB Phase Designs: A Systematic Review.

The current article presents a systematic review of consistency in single-case ABAB phase designs. We applied the CONsistency of DAta Patterns (CONDAP) measure to a sample of 460 data sets retrieved from 119 applied studies published over the past 50 years. The main purpose was to (a) identify typical CONDAP values found in published ABAB designs and (b) develop interpretational guidelines for CONDAP to be used for future studies to assess the consistency of data patterns from similar phases. The overall distribution of CONDAP values is right-skewed with several extreme values to the right of the center of the distribution. The B-phase CONDAP values fall within a narrower range than the A-phase CONDAP values. Based on the cumulative distribution of CONDAP values, we offer the following interpretational guidelines in terms of consistency: very high, 0 ≤ CONDAP ≤ 0.5; high, 0.5 < CONDAP ≤ 1; medium, 1 < CONDAP < 1.5; low, 1.5 < CONDAP ≤ 2; very low, CONDAP > 2. We give examples of combining CONDAP benchmarks with visual analysis of single-case ABAB phase designs and conclude that the majority of data patterns (41.2%) in published ABAB phase designs is medium consistent.

Get Your Head in the Game: A Replicated Single-Case Experimental Design Evaluating the Effect of a Novel Virtual Reality Intervention in People With Chronic Low Back Pain.

Chronic low back pain (CLBP) is a leading cause of disability worldwide. Contemporary treatment of CLBP is suboptimal, with small-moderate effect sizes and high relapse rates. Virtual reality (VR) is an increasingly accessible technology that can improve adherence to exercise programs through gamification. Using VR to facilitate exercise adherence and enjoyment may improve the clinical outcomes. This study aimed to evaluate the effects of a gamified VR graded activity intervention in people with CLBP, using commercially available and bespoke VR programs. A sequentially replicated, multiple-baseline, randomized AB single-case experimental design was undertaken in 10 people with CLBP. Outcomes were assessed daily and included pain intensity (primary) and pain catastrophizing, pain-related fear, and anxiety/worry (secondary). The effect of the intervention on the primary outcome was evaluated using a multilevel-model, nonparametric randomization test. The VR graded activity intervention resulted in a significant reduction in pain intensity (effect estimate = -1.0, standard error = .27, P < .0011) with 4 participants achieving ≥30% pain reduction (minimum important change). There was a significant effect of the intervention on pain catastrophizing but not pain-related fear or anxiety/worry measures. These findings provide preliminary support for a VR graded activity program to reduce pain in people with CLBP. PERSPECTIVE: This novel, VR graded activity intervention reduced pain intensity and catastrophizing in people with CLBP. The intervention also had high adherence and enjoyment. Given that this intervention involved 2 freely available VR programs, it can be easily translated into clinical practice.

A randomized trial of 7-day doripenem versus 10-day imipenem-cilastatin for ventilator-associated pneumonia.

INTRODUCTION: The aim of this study was to compare a 7-day course of doripenem to a 10-day course of imipenem-cilastatin for ventilator-associated pneumonia (VAP) due to Gram-negative bacteria. METHODS: This was a prospective, double-blinded, randomized trial comparing a fixed 7-day course of doripenem one gram as a four-hour infusion every eight hours with a fixed 10-day course of imipenem-cilastatin one gram as a one-hour infusion every eight hours (April 2008 through June 2011). RESULTS: The study was stopped prematurely at the recommendation of the Independent Data Monitoring Committee that was blinded to treatment arm assignment and performed a scheduled review of data which showed signals that were close to the pre-specified stopping limits. The final analyses included 274 randomized patients. The clinical cure rate at the end of therapy (EOT) in the microbiological intent-to-treat (MITT) population was numerically lower for patients in the doripenem arm compared to the imipenem-cilastatin arm (45.6% versus 56.8%; 95% CI, -26.3% to 3.8%). Similarly, the clinical cure rate at EOT was numerically lower for patients with Pseudomonas aeruginosa VAP, the most common Gram-negative pathogen, in the doripenem arm compared to the imipenem-cilastatin arm (41.2% versus 60.0%; 95% CI, -57.2 to 19.5). All cause 28-day mortality in the MITT group was numerically greater for patients in the doripenem arm compared to the imipenem-cilastatin arm (21.5% versus 14.8%; 95% CI, -5.0 to 18.5) and for patients with P. aeruginosa VAP (35.3% versus 0.0%; 95% CI, 12.6 to 58.0). CONCLUSIONS: Among patients with microbiologically confirmed late-onset VAP, a fixed 7-day course of doripenem was found to have non-significant higher rates of clinical failure and mortality compared to a fixed 10-day course of imipenem-cilastatin. Consideration should be given to treating patients with VAP for more than seven days to optimize clinical outcome. TRIAL REGISTRATION: ClinicalTrials.gov: NCT00589693.

Assessing Consistency in Single-Case A-B-A-B Phase Designs.

Previous research has introduced several effect size measures (ESMs) to quantify data aspects of single-case experimental designs (SCEDs): level, trend, variability, overlap, and immediacy. In the current article, we extend the existing literature by introducing two methods for quantifying consistency in single-case A-B-A-B phase designs. The first method assesses the consistency of data patterns across phases implementing the same condition, called CONsistency of DAta Patterns (CONDAP). The second measure assesses the consistency of the five other data aspects when changing from baseline to experimental phase, called CONsistency of the EFFects (CONEFF). We illustrate the calculation of both measures for four A-B-A-B phase designs from published literature and demonstrate how CONDAP and CONEFF can supplement visual analysis of SCED data. Finally, we discuss directions for future research.

Illusion-enhanced Virtual Reality Exercise for Neck Pain: A Replicated Single Case Series.

OBJECTIVES: Body illusions have shown promise in treating some chronic pain conditions. We hypothesized that neck exercises performed in virtual reality (VR) with visual feedback of rotation amplified would reduce persistent neck pain. METHODS: In a multiple-baseline replicated single case series, 8 blinded individuals with persistent neck pain completed a 4-phase intervention (initial n=12, 4 dropouts): (1) "baseline"; (2) "VR" during which participants performed rotation exercises in VR with no manipulation of visual feedback; (3) "VR enhanced" during which identical exercises were performed but visual feedback overstated the range of motion being performed; (4) "follow-up." Primary outcomes were twice-daily measures of pain-free range of motion and pain intensity. During the baseline and follow-up phases, measures were taken but no intervention took place. RESULTS: No differences in primary outcomes were found between VR and baseline, VR enhanced and VR, or VR enhanced and follow-up. DISCUSSION: Our hypothesis, that neck exercises performed in VR with visual feedback of rotation amplified, would reduce persistent neck pain was not supported. Possible explanations and future directions are discussed.

Randomization tests for changing criterion designs.

Randomization tests for alternating treatments designs, multiple baseline designs, and withdrawal/reversal designs are well-established. Recent classifications, however, also mention the "changing criterion design" as a fourth important type of single-case experimental design. In this paper, we examine the potential of randomization tests for changing criterion designs. We focus on the rationale of the randomization test, the random assignment procedure, the choice of the test statistic, and the calculation of randomization test p-values. Two examples using empirical data and an R computer program to perform the calculations are provided. We discuss the problems associated with conceptualizing the changing criterion design as a variant of the multiple baseline design, the potential of the range-bound changing criterion design, experimental control as an all-or-none phenomenon, the necessity of random assignment for the statistical-conclusion validity of the randomization test, and the use of randomization tests in nonrandomized designs.

Comparative Efficacy, Safety and Immunogenicity of Hepavax-Gene TF and Engerix-B Recombinant Hepatitis B Vaccines in Neonates in China.

BACKGROUND: The aim of the study was to compare efficacy, immunogenicity and safety of Hepavax-Gene TF (thimerosal-free) vaccine with comparator in Chinese neonates. METHODS: A double-blind, randomized, parallel-group, stratified study was conducted at multiple sites in China in healthy neonates, consisting of 3 doses of Hepavax-Gene TF or Engerix-B vaccines administered at birth, 1 and 6 months of age, with a 6-month follow-up after vaccination. On the basis of hepatitis B virus (HBV) infection status of mothers, infants were assigned to one of 2 study strata for mothers positive for HBV infection (stratum 1), with or without active replicating virus (substrata 1a, 1b), and for HBV negative mothers (stratum 2). RESULTS: Mother-to-child HBV transmission was prevented in >95% of neonates immunized with Hepavax-Gene TF in stratum 1 at all timepoints and was noninferior to Engerix-B. Seroprotection rates (anti-HBs antibody ≥10 IU/L) at 1 and 6 months postvaccination for Hepavax-Gene TF were over 90% for all exposed neonates. Immunogenicity of Hepavax-Gene was noninferior to Engerix-B except for neonates in substratum 1a at 12 months. Geometric mean concentrations between vaccine groups were not significantly different for neonates at all timepoints except in substratum 1b at 7 months. Both vaccines were well tolerated and had similar local and systemic adverse event profiles. CONCLUSIONS: Hepavax-Gene TF vaccine was equally effective and noninferior to Engerix-B in terms of prevention of mother-to-child HBV transmission in neonates born to mothers positive for hepatitis B surface antigen. Both vaccines elicited seroprotective levels in >90% of all exposed neonates at 12-month follow-up. Both vaccines were well tolerated with similar adverse event profiles.

Phase II study of the efficacy and tolerability of two dosing regimens of the farnesyl transferase inhibitor, R115777, in advanced breast cancer.

PURPOSE: R115777 is an orally active farnesyl transferase inhibitor that specifically blocks farnesylation of proteins involved in growth-factor-dependent cell-signal-transduction pathways. We conducted a phase II study in 76 patients with advanced breast cancer. PATIENTS AND METHODS: Two cohorts of patients were recruited sequentially. The first cohort (n = 41) received a continuous dosing [CD] regimen of R115777 400 or 300 mg bid. The second cohort (n = 35) received 300 mg bid in a cyclical regimen of 21 days of treatment followed by 7 days of rest (intermittent dosing [ID]). RESULTS: In the CD cohort, four patients (10%) had a partial response (PR) and six patients (15%) had stable disease at > or = 24 weeks (SD). In the ID cohort, five patients (14%) had a PR and three patients (9%) had prolonged SD. The first six patients in the CD cohort treated at 400 mg bid all developed grade 3 to 4 neutropenia, so the subsequent 35 patients were treated at 300 mg bid. The incidence of hematologic toxicity was significantly lower in the ID than in the CD (300-mg bid) cohort: grade 3 to 4 neutropenia (14% v 43%; P =.016) and grade 3 to 4 thrombocytopenia (3% v 26%; P =.013). One patient in the ID cohort developed grade 2 to 3 neurotoxicity compared with 15 patients in the CD cohort (3% v 37%; P =.0004). CONCLUSION: The farnesyl transferase inhibitor R115777 has demonstrated clinical activity in patients with metastatic breast cancer, and the ID regimen has a significantly improved therapeutic index compared with the CD regimen.

Trabectedin as a single-agent treatment of advanced breast cancer after anthracycline and taxane treatment: a multicenter, randomized, phase II study comparing 2 administration regimens.

BACKGROUND: The purpose of this study was to assess the efficacy and safety of trabectedin for advanced breast cancer. PATIENTS AND METHODS: In an open-label, phase II, multicenter study, women with advanced breast cancer previously treated with ≤ 2 lines of chemotherapy for advanced disease, including both anthracyclines and taxanes, were randomized (1:1) to 3-hour infusions of trabectedin 1.3 mg/m(2) once every 3 weeks (1/3 treatment arm) or 0.58 mg/m(2) every week for 3 of 4 weeks (3/4 treatment arm). The primary end point was objective response. Secondary end points included time to progression (TTP), progression-free survival (PFS), and overall survival (OS). RESULTS: Fifty-two women (median age, 50 years; median chemotherapy agents, 4) were enrolled. Relative trabectedin dose intensities were 81% and 76% in the 1/3 and 3/4 treatment arms, respectively. Objective response rates were 12% (3 of 25) and 4% (1 of 27), respectively. Stable disease was observed in 14 (56%) and 11 (41%) patients in the 1/3 and 3/4 treatment arms, respectively, with median durations of 3.5 and 3.7 months. Median TTP and PFS were higher in the 1/3 treatment arm (3.1 months each) than in the 3/4 treatment arm (2.0 months each). At a median follow-up of 7 months in both treatment arms, median OS was not reached in the 1/3 treatment arm and was 9.4 months in the 3/4 treatment arm. The most frequent drug-related adverse events in the 1/3 and 3/4 treatment arms, respectively, were alanine aminotransferase (ALT) level increases (68% vs. 63%), nausea (56% vs. 59%), and asthenia (56% vs. 48%). Neutropenia and increases in ALT levels were the most frequent grade 3/4 events. Both types of events were usually transient and reversible. CONCLUSION: In the population studied, trabectedin showed a manageable safety profile for both regimens analyzed. There were higher objective response rates and a longer PFS in the 1/3 treatment arm compared with the 3/4 treatment arm.

A randomised, double-blind trial comparing ceftobiprole medocaril with ceftriaxone with or without linezolid for the treatment of patients with community-acquired pneumonia requiring hospitalisation.

Community-acquired pneumonia (CAP) is a serious infection requiring hospitalisation in 20% of cases. The novel cephalosporin ceftobiprole has microbiological activity against the major bacterial pathogens causing CAP, including Streptococcus pneumoniae, Haemophilus influenzae and Klebsiella pneumoniae, as well as against Staphylococcus aureus, including meticillin-resistant S. aureus (MRSA). This was a multicentre, double-blind study in which 706 patients with CAP severe enough to require hospitalisation were randomised to ceftobiprole or to an expert-recommended course of ceftriaxone ± linezolid (comparator group). Clinical and microbiological outcomes were determined 7-14 days after completion of therapy (test-of-cure visit). For the 469 clinically evaluable patients, cure rates were 86.6% vs. 87.4% for ceftobiprole and comparator, respectively [95% confidence interval (CI) of the difference, -6.9% to 5.3%]; in the intention-to-treat (ITT) analysis of 638 CAP patients, these cure rates were 76.4% vs. 79.3%, respectively (95% CI of the difference, -9.3% to 3.6%). A typical bacterial pathogen was identified in 29% of the ITT population. Microbiological eradication rates in the 144 microbiologically evaluable patients were 88.2% and 90.8% for the respective treatment groups (95% CI of the difference, -12.6% to 7.5%). Both study drugs were well tolerated, with but a minority of patients requiring premature discontinuation due to an adverse event (6% in the ceftobiprole group and 4% in the comparator group). The overall incidence of treatment-related adverse events was higher in the ceftobiprole group, primarily owing to differences in rates of self-limited nausea (7% vs. 2%) and vomiting (5% vs. 2%). In summary, ceftobiprole was non-inferior to the comparator (ceftriaxone ± linezolid) in all clinical and microbiological analyses conducted, suggesting that ceftobiprole has a potential role in treating hospitalised patients with CAP. [ClinicalTrials.gov identifier: NCT00326287].

Patient-reported outcomes as a component of the primary endpoint in a double-blind, placebo-controlled trial in advanced pancreatic cancer.

In this randomized, double-blind, placebo-controlled study comparing gemcitabine+tipifarnib (G+t) or gemcitabine+placebo (G+p) in patients with pancreatic cancer, the primary endpoint of time to deterioration (TTD) was based primarily on patient-reported outcomes. Deterioration was defined as death or worsening of disease-related symptoms, based on patient-reported outcomes of pain intensity and analgesic use in a daily diary, plus investigator-rated weekly performance status. Secondary endpoints included survival and safety. Two hundred and forty-four patients were treated for a total of 4780 weeks, during which the diary was completed daily. Overall, the completion of the diary was found to be feasible: patients completed approximately 95% of scheduled diary entries. Baseline characteristics were well balanced between the two treatment arms. The primary endpoint of TTD was not significantly different between the G+t arm (69 days) and the G+p arm (91 days, P=0.40). Survival was not significantly different between the G+t arm (202 days) and the G+p arm (221 days, P=0.66). The combination of G+t had an acceptable toxicity profile, with primarily neutropenia and thrombocytopenia. Methodologically, measurement of patient-reported outcomes is feasible and useful in assessing the effect of anti-cancer therapy in pancreatic cancer if comprehensive initial and ongoing training is provided to all people involved, including not only the patients but also the study personnel.

Selection models and pattern-mixture models to analyse longitudinal quality of life data subject to drop-out.

Longitudinally observed quality of life data with large amounts of drop-out are analysed. First we used the selection modelling framework, frequently used with incomplete studies. An alternative method consists of using pattern-mixture models. These are also straightforward to implement, but result in a different set of parameters for the measurement and drop-out mechanisms. Since selection models and pattern-mixture models are based upon different factorizations of the joint distribution of measurement and drop-out mechanisms, comparing both models concerning, for example, treatment effect, is a useful form of a sensitivity analysis.

Controlling type I error rate for fast track drug development programmes.

The U.S. Food and Drug Administration (FDA) Modernization Act of 1997 has a Section (No. 112) entitled 'Expediting Study and Approval of Fast Track Drugs' (the Act). In 1998, the FDA issued a 'Guidance for Industry: the Fast Track Drug Development Programs' (the FTDD programmes) to meet the requirement of the Act. The purpose of FTDD programmes is to 'facilitate the development and expedite the review of new drugs that are intended to treat serious or life-threatening conditions and that demonstrate the potential to address unmet medical needs'. Since then many health products have reached patients who suffered from AIDS, cancer, osteoporosis, and many other diseases, sooner by utilizing the Fast Track Act and the FTDD programmes. In the meantime several scientific issues have also surfaced when following the FTDD programmes. In this paper we will discuss the concept of two kinds of type I errors, namely, the 'conditional approval' and the 'final approval' type I errors, and propose statistical methods for controlling them in a new drug submission process.

Strategies to fit pattern-mixture models.

Whereas most models for incomplete longitudinal data are formulated within the selection model framework, pattern-mixture models have gained considerable interest in recent years (Little, 1993, 1994). In this paper, we outline several strategies to fit pattern-mixture models, including the so-called identifying restrictions strategy. Multiple imputation is used to apply this strategy to realistic settings, such as quality-of-life data from a longitudinal study on metastatic breast cancer patients.