Heterogeneity in pragmatic randomised trials: sources and management.

BACKGROUND: Pragmatic trials aim to generate evidence to directly inform patient, caregiver and health-system manager policies and decisions. Heterogeneity in patient characteristics contributes to heterogeneity in their response to the intervention. However, there are many other sources of heterogeneity in outcomes. Based on the expertise and judgements of the authors, we identify different sources of clinical and methodological heterogeneity, which translate into heterogeneity in patient responses-some we consider as desirable and some as undesirable. For each of them, we discuss and, using real-world trial examples, illustrate how heterogeneity should be managed over the whole course of the trial. MAIN TEXT: Heterogeneity in centres and patients should be welcomed rather than limited. Interventions can be flexible or tailored and control interventions are expected to reflect usual care, avoiding use of a placebo. Co-interventions should be allowed; adherence should not be enforced. All these elements introduce heterogeneity in interventions (experimental or control), which has to be welcomed because it mimics reality. Outcomes should be objective and possibly routinely collected; standardised assessment, blinding and adjudication should be avoided as much as possible because this is not how assessment would be done outside a trial setting. The statistical analysis strategy must be guided by the objective to inform decision-making, thus favouring the intention-to-treat principle. Pragmatic trials should consider including process analyses to inform an understanding of the trial results. Needed data to conduct these analyses should be collected unobtrusively. Finally, ethical principles must be respected, even though this may seem to conflict with goals of pragmatism; consent procedures could be incorporated in the flow of care.

Cluster over individual randomization: are study design choices appropriately justified? Review of a random sample of trials.

BACKGROUND: Novel rationales for randomizing clusters rather than individuals appear to be emerging from the push for more pragmatic trials, for example, to facilitate trial recruitment, reduce the costs of research, and improve external validity. Such rationales may be driven by a mistaken perception that choosing cluster randomization lessens the need for informed consent. We reviewed a random sample of published cluster randomized trials involving only individual-level health care interventions to determine (a) the prevalence of reporting a rationale for the choice of cluster randomization; (b) the types of explicit, or if absent, apparent rationales for the use of cluster randomization; (c) the prevalence of reporting patient informed consent for study interventions; and (d) the types of justifications provided for waivers of consent. We considered cluster randomized trials for evaluating exclusively the individual-level health care interventions to focus on clinical trials where individual randomization is only theoretically possible and where there is a general expectation of informed consent. METHODS: A random sample of 40 cluster randomized trials were identified by implementing a validated electronic search filter in two electronic databases (Ovid MEDLINE and Embase), with two reviewers independently extracting information from each trial. Inclusion criteria were the following: primary report of a cluster randomized trial, evaluating exclusively an individual-level health care intervention, published between 2007 and 2016, and conducted in Canada, the United States, European Union, Australia, or low- and middle-income country settings. RESULTS: Twenty-five trials (62.5%, 95% confidence interval = 47.5%-77.5%) reported an explicit rationale for the use of cluster randomization. The most commonly reported rationales were those with logistical or administrative convenience (15 trials, 60%) and those that need to avoid contamination (13 trials, 52%); five trials (20%) were cited rationales related to the push for more pragmatic trials. Twenty-one trials (52.5%, 95% confidence interval = 37%-68%) reported written informed consent for the intervention, two (5%) reported verbal consent, and eight (20%) reported waivers of consent, while in nine trials (22.5%) consent was unclear or not mentioned. Reported justifications for waivers of consent included that study interventions were already used in clinical practice, patients were not randomized individually, and the need to facilitate the pragmatic nature of the trial. Only one trial reported an explicit and appropriate justification for waiver of consent based on minimum criteria in international research ethics guidelines, namely, infeasibility and minimal risk. CONCLUSION: Rationales for adopting cluster over individual randomization and for adopting consent waivers are emerging, related to the need to facilitate pragmatic trials. Greater attention to clear reporting of study design rationales, informed consent procedures, as well as justification for waivers is needed to ensure that such trials meet appropriate ethical standards.

Double-blind randomised controlled trial of vitamin D3 supplementation for the prevention of acute respiratory infection in older adults and their carers (ViDiFlu).

RATIONALE: Low-dose vitamin D supplementation is already recommended in older adults for prevention of fractures and falls, but clinical trials investigating whether higher doses could provide additional protection against acute respiratory infection (ARI) are lacking. OBJECTIVE: To conduct a clinical trial of high-dose versus low-dose vitamin D3 supplementation for ARI prevention in residents of sheltered-accommodation housing blocks ('schemes') and their carers in London, UK. MEASUREMENTS AND METHODS: Fifty-four schemes (137 individual participants) were allocated to the active intervention (vitamin D3 2.4 mg once every 2 months +10 µg daily for residents, 3 mg once every 2 months for carers), and 54 schemes with 103 participants were allocated to control (placebo once every 2 months +vitamin D3 10 µg daily for residents, placebo once every 2 months for carers) for 1 year. Primary outcome was time to first ARI; secondary outcomes included time to first upper/lower respiratory infection (URI/LRI, analysed separately), and symptom duration. MAIN RESULTS: Inadequate vitamin D status was common at baseline: 220/240 (92%) participants had serum 25(OH)D concentration <75 nmol/L. The active intervention did not influence time to first ARI (adjusted HR (aHR) 1.18, 95% CI 0.80 to 1.74, p=0.42). When URI and LRI were analysed separately, allocation to the active intervention was associated with increased risk of URI (aHR 1.48, 95% CI 1.02 to 2.16, p=0.039) and increased duration of URI symptoms (median 7.0 vs 5.0 days for active vs control, adjusted ratio of geometric means 1.34, 95% CI 1.09 to 1.65, p=0.005), but not with altered risk or duration of LRI. CONCLUSIONS: Addition of intermittent bolus-dose vitamin D3 supplementation to a daily low-dose regimen did not influence risk of ARI in older adults and their carers, but was associated with increased risk and duration of URI. TRIAL REGISTRATION NUMBER: clinicaltrials.gov NCT01069874.

Why and when should we cluster randomize?

A cluster randomized trial is defined as a randomized trial in which intact social units of individuals are randomized rather than individuals themselves. Outcomes are observed on individual participants within clusters (such as patients). Such a design allows assessing interventions targeting cluster-level participants (such as physicians), individual participants or both. Indeed, many interventions assessed in cluster randomized trials are actually complex ones, with distinct components targeting different levels. For a cluster-level intervention, cluster randomization is an obvious choice: the intervention is not divisible at the individual-level. For individual-level interventions, cluster randomization may nevertheless be suitable to prevent group contamination, for logistical reasons, to enhance participants' adherence, or when objectives pertain to the cluster level. An unacceptable reason for cluster randomization would be to avoid obtaining individual consent. Indeed, participants in cluster randomized trials have to be protected as in any type of trial design. Participants may be people from whom data are collected, but they may also be people who are intervened upon, and this includes both patients and physicians (for example, physicians receiving training interventions). Consent should be sought as soon as possible, although there may exist situations where participants may consent only for data collection, not for being exposed to the intervention (because, for instance, they cannot opt-out). There may even be situations where participants are not able to consent at all. In this latter situation a waiver of consent must be granted by a research ethics committee.

High-dose vitamin D(3) during intensive-phase antimicrobial treatment of pulmonary tuberculosis: a double-blind randomised controlled trial.

BACKGROUND: Vitamin D was used to treat tuberculosis in the pre-antibiotic era, and its metabolites induce antimycobacterial immunity in vitro. Clinical trials investigating the effect of adjunctive vitamin D on sputum culture conversion are absent. METHODS: We undertook a multicentre randomised controlled trial of adjunctive vitamin D in adults with sputum smear-positive pulmonary tuberculosis in London, UK. 146 patients were allocated to receive 2·5 mg vitamin D(3) or placebo at baseline and 14, 28, and 42 days after starting standard tuberculosis treatment. The primary endpoint was time from initiation of antimicrobial treatment to sputum culture conversion. Patients were genotyped for TaqI and FokI polymorphisms of the vitamin D receptor, and interaction analyses were done to assess the influence of the vitamin D receptor genotype on response to vitamin D(3). This trial is registered with ClinicalTrials.gov number NCT00419068. FINDINGS: 126 patients were included in the primary efficacy analysis (62 assigned to intervention, 64 assigned to placebo). Median time to sputum culture conversion was 36·0 days in the intervention group and 43·5 days in the placebo group (adjusted hazard ratio 1·39, 95% CI 0·90-2·16; p=0.14). TaqI genotype modified the effect of vitamin D supplementation on time to sputum culture conversion (p(interaction)=0·03), with enhanced response seen only in patients with the tt genotype (8·09, 95% CI 1·36-48·01; p=0·02). FokI genotype did not modify the effect of vitamin D supplementation (p(interaction)=0·85). Mean serum 25-hydroxyvitamin D concentration at 56 days was 101·4 nmol/L in the intervention group and 22·8 nmol/L in the placebo group (95% CI for difference 68·6-88·2; p<0·0001). INTERPRETATION: Administration of four doses of 2·5 mg vitamin D(3) increased serum 25-hydroxyvitamin D concentrations in patients receiving intensive-phase treatment for pulmonary tuberculosis. Vitamin D did not significantly affect time to sputum culture conversion in the whole study population, but it did significantly hasten sputum culture conversion in participants with the tt genotype of the TaqI vitamin D receptor polymorphism. FUNDING: British Lung Foundation.

An analysis of published trials found that current use of pragmatic trial labels is uninformative.

OBJECTIVES: Randomized trials labelled as "pragmatic" are attractive to funders, patients, and clinicians as the label implies that the results are directly applicable to clinical care. We examined how authors justify use of the label (e.g., by referring to one or more PRECIS [PRagmatic Explanatory Continuum Indicator Summary]-2 domains). STUDY DESIGN AND SETTING: We reviewed primary trial reports published 2014-2019, registered in ClinicalTrials.gov and using the pragmatic label anywhere in the report. RESULTS: Among 415 trials, the label was justified by reference to at least one design element in 282 (68.0%); of these, 240 (85.1%) referenced trial characteristics that can be mapped to one or more of the PRECIS-2 domains, most commonly eligibility (91, 32.3%), setting (90, 31.9%), flexibility delivery (89, 31.6%), and organization (75, 26.6%); 42 (14.9%) referenced characteristics that are not PRECIS-2 domains, most commonly type of intervention/comparator (48, 17%), recruitment without consent (22, 7.8%), routinely collected data (22, 7.8%), and cluster randomization (20, 7.1%). Most reports referenced only one or two design elements. Overall, 9/415 (2.2%) provided PRECIS wheels. CONCLUSION: Current use of pragmatic labels is uninformative. Authors should clarify the decision the trial is intended to support and include a PRECIS-2 table to make the design transparent.

Withdrawal of inhaled corticosteroids in individuals with COPD--a systematic review and comment on trial methodology.

Inhaled corticosteroids (ICS) reduce COPD exacerbation frequency and slow decline in health related quality of life but have little effect on lung function, do not reduce mortality, and increase the risk of pneumonia. We systematically reviewed trials in which ICS have been withdrawn from patients with COPD, with the aim of determining the effect of withdrawal, understanding the differing results between trials, and making recommendations for improving methodology in future trials where medication is withdrawn. Trials were identified by two independent reviewers using MEDLINE, EMBASE and CINAHL, citations of identified studies were checked, and experts contacted to identify further studies. Data extraction was completed independently by two reviewers. The methodological quality of each trial was determined by assessing possible sources of systematic bias as recommended by the Cochrane collaboration. We included four trials; the quality of three was adequate. In all trials, outcomes were generally worse for patients who had had ICS withdrawn, but differences between outcomes for these patients and patients who continued with medication were mostly small and not statistically significant. Due to data paucity we performed only one meta-analysis; this indicated that patients who had had medication withdrawn were 1.11 (95% CI 0.84 to 1.46) times more likely to have an exacerbation in the following year, but the definition of exacerbations was not consistent between the three trials, and the impact of withdrawal was smaller in recent trials which were also trials conducted under conditions that reflected routine practice. There is no evidence from this review that withdrawing ICS in routine practice results in important deterioration in patient outcomes. Furthermore, the extent of increase in exacerbations depends on the way exacerbations are defined and managed and may depend on the use of other medication. In trials where medication is withdrawn, investigators should report other medication use, definitions of exacerbations and management of patients clearly. Intention to treat analyses should be used and interpreted appropriately.

Association of intracluster correlation measures with outcome prevalence for binary outcomes in cluster randomised trials.

In cluster randomised trials, a measure of intracluster correlation such as the intraclass correlation coefficient (ICC) should be reported for each primary outcome. Providing intracluster correlation estimates may help in calculating sample size of future cluster randomised trials and also in interpreting the results of the trial from which they are derived. For a binary outcome, the ICC is known to be associated with its prevalence, which raises at least two issues. First, it questions the use of ICC estimates obtained on a binary outcome in a trial for sample size calculations in a subsequent trial in which the same binary outcome is expected to have a different prevalence. Second, it challenges the interpretation of ICC estimates because they do not solely depend on clustering level. Other intracluster correlation measures proposed for clustered binary data settings include the variance partition coefficient, the median odds ratio and the tetrachoric correlation coefficient. Under certain assumptions, the theoretical maximum possible value for an ICC associated with a binary outcome can be derived, and we proposed the relative deviation of an ICC estimate to this maximum value as another measure of the intracluster correlation. We conducted a simulation study to explore the dependence of these intracluster correlation measures on outcome prevalence and found that all are associated with prevalence. Even if all depend on prevalence, the tetrachoric correlation coefficient computed with Kirk's approach was less dependent on the outcome prevalence than the other measures when the intracluster correlation was about 0.05. We also observed that for lower values, such as 0.01, the analysis of variance estimator of the ICC is preferred.

Pilot and feasibility studies for pragmatic trials have unique considerations and areas of uncertainty.

BACKGROUND AND OBJECTIVE: Feasibility studies are increasingly being used to support the development of, and investigate uncertainties around, future large-scale trials. The future trial can be designed with either a pragmatic or explanatory mindset. Whereas pragmatic trials aim to inform the choice between different care options and thus, are designed to resemble conditions outside of a clinical trial environment, explanatory trials examine the benefit of a treatment under more controlled conditions. There is existing guidance for designing feasibility studies, but none that explicitly considers the goals of pragmatic designs. We aimed to identify unique areas of uncertainty that are relevant to planning a pragmatic trial. RESULTS: We identified ten relevant domains, partly based on the pragmatic-explanatory continuum indicator summary-2 (PRECIS-2) framework, and describe potential questions of uncertainty within each: intervention development, research ethics, participant identification and eligibility, recruitment of individuals, setting, organization, flexibility of delivery, flexibility of adherence, follow-up, and importance of primary outcome to patients and decision-makers. We present examples to illustrate how uncertainty in these domains might be addressed within a feasibility study. CONCLUSION: Researchers planning a feasibility study in advance of a pragmatic trial should consider feasibility objectives specifically relevant to areas of uncertainty for pragmatic trials.

Progression from external pilot to definitive randomised controlled trial: a methodological review of progression criteria reporting.

OBJECTIVES: Prespecified progression criteria can inform the decision to progress from an external randomised pilot trial to a definitive randomised controlled trial. We assessed the characteristics of progression criteria reported in external randomised pilot trial protocols and results publications, including whether progression criteria were specified a priori and mentioned in prepublication peer reviewer reports. STUDY DESIGN: Methodological review. METHODS: We searched four journals through PubMed: British Medical Journal Open, Pilot and Feasibility Studies, Trials and Public Library of Science One. Eligible publications reported external randomised pilot trial protocols or results, were published between January 2018 and December 2019 and reported progression criteria. We double data extracted 25% of the included publications. Here we report the progression criteria characteristics. RESULTS: We included 160 publications (123 protocols and 37 completed trials). Recruitment and retention were the most frequent indicators contributing to progression criteria. Progression criteria were mostly reported as distinct thresholds (eg, achieving a specific target; 133/160, 83%). Less than a third of the planned and completed pilot trials that included qualitative research reported how these findings would contribute towards progression criteria (34/108, 31%). The publications seldom stated who established the progression criteria (12/160, 7.5%) or provided rationale or justification for progression criteria (44/160, 28%). Most completed pilot trials reported the intention to proceed to a definitive trial (30/37, 81%), but less than half strictly met all of their progression criteria (17/37, 46%). Prepublication peer reviewer reports were available for 153/160 publications (96%). Peer reviewer reports for 86/153 (56%) publications mentioned progression criteria, with peer reviewers of 35 publications commenting that progression criteria appeared not to be specified. CONCLUSIONS: Many external randomised pilot trial publications did not adequately report or propose prespecified progression criteria to inform whether to proceed to a future definitive randomised controlled trial.

Transparency of informed consent in pilot and feasibility studies is inadequate: a single-center quality assurance study.

BACKGROUND: Pilot and feasibility studies (PAFS) often have complex objectives aimed at assessing feasibility of conducting a larger study. These may not be clear to participants in pilot studies. METHODS: Here, we aimed to assess the transparency of informed consent in PAFS by investigating whether researchers communicate, through patient information leaflets and consent forms, key features of the studies. We collected this data from original versions of these documents submitted for ethics approval and the final approved documents for PAFS submitted to the Hamilton Integrated Research Ethics Board, Canada. RESULTS: One hundred eighty-four PAFS, submitted for ethics approval from 2004 to 2020, were included, and we found that of the approved consent documents which were provided to participants, 83.2% (153) stated the terms "pilot" or "feasibility" in their title, 12% (22) stated the definition of a pilot/feasibility study, 42.4% (78) of the studies stated their intent to assess feasibility, 19.6% (36) stated the specific feasibility objectives, 1.6% (3) stated the criteria for success of the pilot study, and 0.5% (1) stated all five of these criteria. After ethics review, a small increase in transparency occurred, ranging from 1.6 to 2.8% depending on the criteria. By extracting data from the protocols of the PAFS, we found that 73.9% (136) stated intent to assess feasibility, 71.2% (131) stated specific feasibility objectives, and 33.7% (62) stated criteria for success of the study to lead to a larger study. CONCLUSION: The transparency of informed consent in PAFS is inadequate and needs to be specifically addressed by research ethics guidelines. Research ethics boards and researchers ought to be made aware and mindful of best practices of informed consent in the context of PAFS.

Is the R coefficient of interest in cluster randomized trials with a binary outcome?

In cluster randomized trials, the intraclass correlation coefficient (ICC) is classically used to measure clustering. When the outcome is binary, the ICC is known to be associated with the prevalence of the outcome. This association challenges its interpretation and can be problematic for sample size calculation. To overcome these situations, Crespi et al. extended a coefficient named R, initially proposed by Rosner for ophthalmologic data, to cluster randomized trials. Crespi et al. asserted that R may be less influenced by the outcome prevalence than is the ICC, although the authors provided only empirical data to support their assertion. They also asserted that "the traditional ICC approach to sample size determination tends to overpower studies under many scenarios, calling for more clusters than truly required", although they did not consider empirical power. The aim of this study was to investigate whether R could indeed be considered independent of the outcome prevalence. We also considered whether sample size calculation should be better based on the R coefficient or the ICC. Considering the particular case of 2 individuals per cluster, we theoretically demonstrated that R is not symmetrical around the 0.5 prevalence value. This in itself demonstrates the dependence of R on prevalence. We also conducted a simulation study to explore the case of both fixed and variable cluster sizes greater than 2. This simulation study demonstrated that R decreases when prevalence increases from 0 to 1. Both the analytical and simulation results demonstrate that R depends on the outcome prevalence. In terms of sample size calculation, we showed that an approach based on the ICC is preferable to an approach based on the R coefficient because with the former, the empirical power is closer to the nominal one. Hence, the R coefficient does not outperform the ICC for binary outcomes because it does not offer any advantage over the ICC.

Assessing the transparency of informed consent in feasibility and pilot studies: a single-centre quality assurance study protocol.

INTRODUCTION: Pilot/feasibility studies assess the feasibility of conducting a larger study. Although researchers ought to communicate the feasibility objectives to their participants, many research ethics guidelines do not comment on how informed consent applies to pilot studies. It is unclear whether researchers and research ethics boards clearly communicate the purpose of pilot studies to participants consenting.The primary objective of this study is to assess whether pilot/feasibility studies submitted for ethics approval to a research ethics board transparently communicate the purpose of the study to participants through their informed consent practice. A highly transparent consent practice entails the consent documents communicate: (1) the term 'pilot' or 'feasibility' in the title; (2) the definition of a pilot/feasibility study; (3) the primary objectives of the study are to assess feasibility; (4) the specific feasibility objectives; and (5) the criteria for the study to successfully lead to the main study. The secondary objectives are to assess whether there is a difference between submitted and revised versions of the consent documents (revisions are made to obtain research ethics approval), to determine factors associated with transparent consent practices and to assess the consistency with which pilot and feasibility studies assess feasibility outcomes as their primary objectives. METHODS AND ANALYSIS: This is a retrospective review of informed consent information for pilot/feasibility studies submitted to the Hamilton integrated Research Ethics Board, Canada. We will look at submitted and revised consent documents for pilot/feasibility studies submitted over a 14-year period. We will use descriptive statistics to summarise data, reporting results as percentages with 95% CIs, and conduct logistic regression to determine characteristics associated with transparent consent practices. ETHICS AND DISSEMINATION: The study protocol was approved by the Hamilton integrated Research Ethics Board, and the results of this study will be submitted for publication in a peer-reviewed journal.

Ethical implications of excessive cluster sizes in cluster randomised trials.

The cluster randomised trial (CRT) is commonly used in healthcare research. It is the gold-standard study design for evaluating healthcare policy interventions. A key characteristic of this design is that as more participants are included, in a fixed number of clusters, the increase in achievable power will level off. CRTs with cluster sizes that exceed the point of levelling-off will have excessive numbers of participants, even if they do not achieve nominal levels of power. Excessively large cluster sizes may have ethical implications due to exposing trial participants unnecessarily to the burdens of both participating in the trial and the potential risks of harm associated with the intervention. We explore these issues through the use of two case studies. Where data are routinely collected, available at minimum cost and the intervention poses low risk, the ethical implications of excessively large cluster sizes are likely to be low (case study 1). However, to maximise the social benefit of the study, identification of excessive cluster sizes can allow for prespecified and fully powered secondary analyses. In the second case study, while there is no burden through trial participation (because the outcome data are routinely collected and non-identifiable), the intervention might be considered to pose some indirect risk to patients and risks to the healthcare workers. In this case study it is therefore important that the inclusion of excessively large cluster sizes is justifiable on other grounds (perhaps to show sustainability). In any randomised controlled trial, including evaluations of health policy interventions, it is important to minimise the burdens and risks to participants. Funders, researchers and research ethics committees should be aware of the ethical issues of excessively large cluster sizes in cluster trials.

Quality of reporting of pilot and feasibility cluster randomised trials: a systematic review.

OBJECTIVES: To systematically review the quality of reporting of pilot and feasibility of cluster randomised trials (CRTs). In particular, to assess (1) the number of pilot CRTs conducted between 1 January 2011 and 31 December 2014, (2) whether objectives and methods are appropriate and (3) reporting quality. METHODS: We searched PubMed (2011-2014) for CRTs with 'pilot' or 'feasibility' in the title or abstract; that were assessing some element of feasibility and showing evidence the study was in preparation for a main effectiveness/efficacy trial. Quality assessment criteria were based on the Consolidated Standards of Reporting Trials (CONSORT) extensions for pilot trials and CRTs. RESULTS: Eighteen pilot CRTs were identified. Forty-four per cent did not have feasibility as their primary objective, and many (50%) performed formal hypothesis testing for effectiveness/efficacy despite being underpowered. Most (83%) included 'pilot' or 'feasibility' in the title, and discussed implications for progression from the pilot to the future definitive trial (89%), but fewer reported reasons for the randomised pilot trial (39%), sample size rationale (44%) or progression criteria (17%). Most defined the cluster (100%), and number of clusters randomised (94%), but few reported how the cluster design affected sample size (17%), whether consent was sought from clusters (11%), or who enrolled clusters (17%). CONCLUSIONS: That only 18 pilot CRTs were identified necessitates increased awareness of the importance of conducting and publishing pilot CRTs and improved reporting. Pilot CRTs should primarily be assessing feasibility, avoiding formal hypothesis testing for effectiveness/efficacy and reporting reasons for the pilot, sample size rationale and progression criteria, as well as enrolment of clusters, and how the cluster design affects design aspects. We recommend adherence to the CONSORT extensions for pilot trials and CRTs.

Asking older people about fear of falling did not have a negative effect.

BACKGROUND AND OBJECTIVE: To assess whether completing a questionnaire on risk of falling could affect outcome measures: fear of falling, reported falls, and health service contacts in older people (panel conditioning). METHODS: We used a postal questionnaire to assess the effect on falls risk of implementing falls injury prevention guidelines within a single locality in outer London, UK. We compared responses for the baseline and 6-month follow-up surveys with those for a fresh survey. The latter was sent to a new pool of subjects drawn from the same population, and was sent only once; timing coincided with the follow-up survey. RESULTS: At baseline, we received 498 responses for 1,000 (50%) surveys sent; of these, 358 (72%) subsequently returned the follow-up survey. For the fresh survey, we received 1,261 out of 2,000 (61%) responses to the fresh survey. The odds ratio for the effect of panel conditioning on fear of falling was 0.92 (95% confidence interval CI = 0.64-1.33), within our predefined limit for equivalence. Odds ratios for the effect on reported falls and health service contacts were 0.87 (95% CI = 0.59-1.29) and 0.75 (95% CI = 0.55-1.02), respectively. CONCLUSION: The proportions of subjects who feared falling in the follow-up survey and in the fresh survey were equivalent. Reduced reporting of falls and health service use in the follow-up survey suggest that the potential for panel effects cannot be ignored.

Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method.

BACKGROUND: Cluster randomized trials are increasingly popular. In many of these trials, cluster sizes are unequal. This can affect trial power, but standard sample size formulae for these trials ignore this. Previous studies addressing this issue have mostly focused on continuous outcomes or methods that are sometimes difficult to use in practice. METHODS: We show how a simple formula can be used to judge the possible effect of unequal cluster sizes for various types of analyses and both continuous and binary outcomes. We explore the practical estimation of the coefficient of variation of cluster size required in this formula and demonstrate the formula's performance for a hypothetical but typical trial randomizing UK general practices. RESULTS: The simple formula provides a good estimate of sample size requirements for trials analysed using cluster-level analyses weighting by cluster size and a conservative estimate for other types of analyses. For trials randomizing UK general practices the coefficient of variation of cluster size depends on variation in practice list size, variation in incidence or prevalence of the medical condition under examination, and practice and patient recruitment strategies, and for many trials is expected to be approximately 0.65. Individual-level analyses can be noticeably more efficient than some cluster-level analyses in this context. CONCLUSIONS: When the coefficient of variation is <0.23, the effect of adjustment for variable cluster size on sample size is negligible. Most trials randomizing UK general practices and many other cluster randomized trials should account for variable cluster size in their sample size calculations.

How big should the pilot study for my cluster randomised trial be?

There is currently a lot of interest in pilot studies conducted in preparation for randomised controlled trials. This paper focuses on sample size requirements for external pilot studies for cluster randomised trials. We consider how large an external pilot study needs to be to assess key parameters for input to the main trial sample size calculation when the primary outcome is continuous, and to estimate rates, for example recruitment rates, with reasonable precision. We used simulation to provide the distribution of the expected number of clusters for the main trial under different assumptions about the natural cluster size, intra-cluster correlation, eventual cluster size in the main trial, and various decisions made at the piloting stage. We chose intra-cluster correlation values and pilot study size to reflect those commonly reported in the literature. Our results show that estimates of sample size required for the main trial are likely to be biased downwards and very imprecise unless the pilot study includes large numbers of clusters and individual participants. We conclude that pilot studies will usually be too small to estimate parameters required for estimating a sample size for a main cluster randomised trial (e.g. the intra-cluster correlation coefficient) with sufficient precision and too small to provide reliable estimates of rates for process measures such as recruitment or follow-up rates.

Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework.

We describe a framework for defining pilot and feasibility studies focusing on studies conducted in preparation for a randomised controlled trial. To develop the framework, we undertook a Delphi survey; ran an open meeting at a trial methodology conference; conducted a review of definitions outside the health research context; consulted experts at an international consensus meeting; and reviewed 27 empirical pilot or feasibility studies. We initially adopted mutually exclusive definitions of pilot and feasibility studies. However, some Delphi survey respondents and the majority of open meeting attendees disagreed with the idea of mutually exclusive definitions. Their viewpoint was supported by definitions outside the health research context, the use of the terms 'pilot' and 'feasibility' in the literature, and participants at the international consensus meeting. In our framework, pilot studies are a subset of feasibility studies, rather than the two being mutually exclusive. A feasibility study asks whether something can be done, should we proceed with it, and if so, how. A pilot study asks the same questions but also has a specific design feature: in a pilot study a future study, or part of a future study, is conducted on a smaller scale. We suggest that to facilitate their identification, these studies should be clearly identified using the terms 'feasibility' or 'pilot' as appropriate. This should include feasibility studies that are largely qualitative; we found these difficult to identify in electronic searches because researchers rarely used the term 'feasibility' in the title or abstract of such studies. Investigators should also report appropriate objectives and methods related to feasibility; and give clear confirmation that their study is in preparation for a future randomised controlled trial designed to assess the effect of an intervention.