Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects.

OBJECTIVES: This study aims to (1) elucidate whether the Hawthorne effect exists, (2) explore under what conditions, and (3) estimate the size of any such effect. STUDY DESIGN AND SETTING: This systematic review summarizes and evaluates the strength of available evidence on the Hawthorne effect. An inclusive definition of any form of research artifact on behavior using this label, and without cointerventions, was adopted. RESULTS: Nineteen purposively designed studies were included, providing quantitative data on the size of the effect in eight randomized controlled trials, five quasiexperimental studies, and six observational evaluations of reporting on one's behavior by answering questions or being directly observed and being aware of being studied. Although all but one study was undertaken within health sciences, study methods, contexts, and findings were highly heterogeneous. Most studies reported some evidence of an effect, although significant biases are judged likely because of the complexity of the evaluation object. CONCLUSION: Consequences of research participation for behaviors being investigated do exist, although little can be securely known about the conditions under which they operate, their mechanisms of effects, or their magnitudes. New concepts are needed to guide empirical studies.

Investigating the relationship between predictability and imbalance in minimisation: a simulation study.

BACKGROUND: The use of restricted randomisation methods such as minimisation is increasing. This paper investigates under what conditions it is preferable to use restricted randomisation in order to achieve balance between treatment groups at baseline with regard to important prognostic factors and whether trialists should be concerned that minimisation may be considered deterministic. METHODS: Using minimisation as the randomisation algorithm, treatment allocation was simulated for hypothetical patients entering a theoretical study having values for prognostic factors randomly assigned with a stipulated probability. The number of times the allocation could have been determined with certainty and the imbalances which might occur following randomisation using minimisation were examined. RESULTS: Overall treatment balance is relatively unaffected by reducing the probability of allocation to optimal treatment group (P) but within-variable balance can be affected by any P <1. This effect is magnified by increased numbers of prognostic variables, the number of categories within them and the prevalence of these categories within the study population. CONCLUSIONS: In general, for smaller trials, probability of treatment allocation to the treatment group with fewer numbers requires a larger value P to keep treatment and variable groups balanced. For larger trials probability of allocation values from P = 0.5 to P = 0.8 can be used while still maintaining balance. For one prognostic variable there is no significant benefit in terms of predictability in reducing the value of P. However, for more than one prognostic variable, significant reduction in levels of predictability can be achieved with the appropriate choice of P for the given trial design.

Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement.

The CONSORT (Consolidated Standards of Reporting Trials) Statement, which includes a checklist and a flow diagram, is a guideline developed to help authors improve the reporting of the findings from randomized controlled trials. It was updated most recently in 2010. Its primary focus is on individually randomized trials with 2 parallel groups that assess the possible superiority of one treatment compared with another. The CONSORT Statement has been extended to other trial designs such as cluster randomization, and recommendations for noninferiority and equivalence trials were made in 2006. In this article, we present an updated extension of the CONSORT checklist for reporting noninferiority and equivalence trials, based on the 2010 version of the CONSORT Statement and the 2008 CONSORT Statement for the reporting of abstracts, and provide illustrative examples and explanations for those items that differ from the main 2010 CONSORT checklist. The intent is to improve reporting of noninferiority and equivalence trials, enabling readers to assess the reliability of their results and conclusions.

Use of randomisation in clinical trials: a survey of UK practice.

BACKGROUND: In healthcare research the randomised controlled trial is seen as the gold standard because it ensures selection bias is minimised. However, there is uncertainty as to which is the most preferred method of randomisation in any given setting and to what extent more complex methods are actually being implemented in the field. METHODS: In this paper we describe the results of a survey of UK academics and publicly funded researchers to examine the extent of the use of various methods of randomisation in clinical trials. RESULTS: Trialists reported using simple randomisation, permuted blocks and stratification more often than more complex methods such as minimisation. Most trialists believed that simple randomisation is suitable for larger trials but there is a high probability of possible imbalance between treatment groups in small trials. It was thought that groups should be balanced at baseline to avoid imbalance and help face-validity. However, very few respondents considered that more complex methods offer any advantages. CONCLUSIONS: This paper demonstrates that for most UK trialists the preferred method of randomisation is using permuted blocks of varying random length within strata. This method eliminates the problem of predictability while maintaining balance across combinations of factors. If the number of prognostic factors is large, then minimisation can be used to provide treatment balance as well as balance over these factors. However, only those factors known to affect outcome should be considered.

Methods of data collection and analysis for the economic evaluation alongside a national, multi-centre trial in the UK: conventional ventilation or ECMO for Severe Adult Respiratory Failure (CESAR).

BACKGROUND: Extracorporeal Membrane Oxygenation (ECMO) is a technology used in treatment of patients with severe but potentially reversible respiratory failure. A multi-centre randomised controlled trial (CESAR) was funded in the UK to compare care including ECMO with conventional intensive care management. The protocol and funding for the CESAR trial included plans for economic data collection and analysis. Given the high cost of treatment, ECMO is considered an expensive technology for many funding systems. However, conventional treatment for severe respiratory failure is also one of the more costly forms of care in any health system. METHODS/DESIGN: The objectives of the economic evaluation are to compare the costs of a policy of referral for ECMO with those of conventional treatment; to assess cost-effectiveness and the cost-utility at 6 months follow-up; and to assess the cost-utility over a predicted lifetime. Resources used by patients in the trial are identified. Resource use data are collected from clinical report forms and through follow up interviews with patients. Unit costs of hospital intensive care resources are based on parallel research on cost functions in UK NHS intensive care units. Other unit costs are based on published NHS tariffs. Cost effectiveness analysis uses the outcome: survival without severe disability. Cost utility analysis is based on quality adjusted life years gained based on the Euroqol EQ-5D at 6 months. Sensitivity analysis is planned to vary assumptions about transport costs and method of costing intensive care. Uncertainty will also be expressed in analysis of individual patient data. Probabilities of cost effectiveness given different funding thresholds will be estimated. DISCUSSION: In our view it is important to record our methods in detail and present them before publication of the results of the trial so that a record of detail not normally found in the final trial reports can be made available in the public domain. TRIAL REGISTRATIONS: The CESAR trial registration number is ISRCTN47279827.

Marketing and clinical trials: a case study.

BACKGROUND: Publicly funded clinical trials require a substantial commitment of time and money. To ensure that sufficient numbers of patients are recruited it is essential that they address important questions in a rigorous manner and are managed well, adopting effective marketing strategies. METHODS: Using methods of analysis drawn from management studies, this paper presents a structured assessment framework or reference model, derived from a case analysis of the MRC's CRASH trial, of 12 factors that may affect the success of the marketing and sales activities associated with clinical trials. RESULTS: The case study demonstrates that trials need various categories of people to buy in - hence, to be successful, trialists must embrace marketing strategies to some extent. CONCLUSION: The performance of future clinical trials could be enhanced if trialists routinely considered these factors.

Financial considerations in the conduct of multi-centre randomised controlled trials: evidence from a qualitative study.

BACKGROUND: Securing and managing finances for multicentre randomised controlled trials is a highly complex activity which is rarely considered in the research literature. This paper describes the process of financial negotiation and the impact of financial considerations in four UK multicentre trials. These trials had met, or were on schedule to meet, recruitment targets agreed with their public-sector funders. The trials were considered within a larger study examining factors which might be associated with trial recruitment (STEPS). METHODS: In-depth semi-structured telephone interviews were conducted in 2003-04 with 45 individuals with various responsibilities to one of the four trials. Interviewees were recruited through purposive and then snowball sampling. Interview transcripts were analysed with the assistance of the qualitative package Atlas-ti. RESULTS: The data suggest that the UK system of dividing funds into research, treatment and NHS support costs brought the trial teams into complicated negotiations with multiple funders. The divisions were somewhat malleable and the funding system was used differently in each trial. The fact that all funders had the potential to influence and shape the trials considered here was an important issue as the perspectives of applicants and funders could diverge. The extent and range of industry involvement in non-industry-led trials was striking. Three broad periods of financial work (foundation, maintenance, and resourcing completion) were identified. From development to completion of a trial, the trialists had to be resourceful and flexible, adapting to changing internal and external circumstances. In each period, trialists and collaborators could face changing costs and challenges. Each trial extended the recruitment period; three required funding extensions from MRC or HTA. CONCLUSION: This study highlights complex financial aspects of planning and conducting trials, especially where multiple funders are involved. Recognition of the importance of financial stability and of the need for appropriate training in this area should be paralleled by further similar research with a broader range of trials, aimed at understanding and facilitating the conduct of clinical research.

What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies.

BACKGROUND: A commonly reported problem with the conduct of multicentre randomised controlled trials (RCTs) is that recruitment is often slower or more difficult than expected, with many trials failing to reach their planned sample size within the timescale and funding originally envisaged. The aim of this study was to explore factors that may have been associated with good and poor recruitment in a cohort of multicentre trials funded by two public bodies: the UK Medical Research Council (MRC) and the Health Technology Assessment (HTA) Programme. METHODS: The cohort of trials was identified from the administrative databases held by the two funding bodies. 114 trials that recruited participants between 1994 and 2002 met the inclusion criteria. The full scientific applications and subsequent trial reports submitted by the trial teams to the funders provided the principal data sources. Associations between trial characteristics and recruitment success were tested using the Chi-squared test, or Fisher's exact test where appropriate. RESULTS: Less than a third (31%) of the trials achieved their original recruitment target and half (53%) were awarded an extension. The proportion achieving targets did not appear to improve over time. The overall start to recruitment was delayed in 47 (41%) trials and early recruitment problems were identified in 77 (63%) trials. The inter-relationship between trial features and recruitment success was complex. A variety of strategies were employed to try to increase recruitment, but their success could not be assessed. CONCLUSION: Recruitment problems are complex and challenging. Many of the trials in the cohort experienced recruitment difficulties. Trials often required extended recruitment periods (sometimes supported by additional funds). While this is of continuing concern, success in addressing the trial question may be more important than recruitment alone.

Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement.

The CONSORT (Consolidated Standards of Reporting Trials) Statement, including a checklist and a flow diagram, was developed to help authors improve their reporting of randomized controlled trials. Its primary focus was on individually randomized trials with 2 parallel groups that assess the possible superiority of one treatment compared with another but is now being extended to other trial designs. Noninferiority and equivalence trials have methodological features that differ from superiority trials and present particular difficulties in design, conduct, analysis, and interpretation. Although the rationale for such trials occurs frequently, those designed and described specifically as noninferiority or equivalence trials appear less commonly in the medical literature. The quality of reporting of those that are published is often inadequate. In this article, we present an adapted CONSORT checklist for reporting noninferiority and equivalence trials and provide illustrative examples and explanations for those items amended from the original CONSORT checklist. The intent is to improve reporting of noninferiority and equivalence trials, enabling readers to assess the validity of their results and conclusions.

Ensayos clínicos aleatorizados comunitarios (CONSORT CLUSTER) / The CONSORT statement for cluster randomised trials

Tras la publicación de la declaración CONSORT, actualmente hay un mayor conocimiento de la necesidad de publicar adecuadamente los resultados que se obtienen en los ensayos clínicos controlados y aleatorizados. La declaración CONSORT incluye una lista de aspectos que se debe considerar en la publicación del ensayo clínico. La declaración CONSORT original se desarrolló para conseguir la adecuada publicación de los ensayos clínicos controlados, aleatorizados de grupos paralelos en los que se asignan a los distintos grupos de intervención participantes individuales. Sin embargo, en los ensayos clínicos aleatorizados comunitarios, a los distintos grupos de estudio se asignan aleatoriamente grupos de participantes más que individuos. El proceso de asignación de grupos de participantes plantea consideraciones adicionales respecto de la publicación de los resultados y ha dado lugar a la publicación de una ampliación de la declaración CONSORT dirigida específicamente a la consideración de este tipo de ensayos. En este artículo se revisará la ampliación de la declaración CONSORT referida a los ensayos clínicos aleatorizados comunitarios, con hincapié en características especiales (AU)

With the publication of the CONSORT statement there is now increased awareness of the need to adequately report the findings of randomised controlled trials. The CONSORT statement includes a checklist of items that should be addressed in the trial report. The original CONSORT statement was developed to ensure the appropriate reporting of parallel group randomised controlled trials in which individual participants are allocated to different intervention groups. In cluster randomised trials, however, groups of participants, rather than individuals, are randomly allocated to study groups. The process of allocating groups of participants raises additional reporting considerations and led to the publication of an extension to the CONSORT statement specifically for cluster randomised trials. In this paper we review the CONSORT extension to cluster randomised trials, outlining the special features of the cluster randomised trial which must be considered (AU)

[The CONSORT statement for cluster randomised trials]. / Ensayos clínicos aleatorizados comunitarios (CONSORT CLUSTER).

With the publication of the CONSORT statement there is now increased awareness of the need to adequately report the findings of randomised controlled trials. The CONSORT statement includes a checklist of items that should be addressed in the trial report. The original CONSORT statement was developed to ensure the appropriate reporting of parallel group randomised controlled trials in which individual participants are allocated to different intervention groups. In cluster randomised trials, however, groups of participants, rather than individuals, are randomly allocated to study groups. The process of allocating groups of participants raises additional reporting considerations and led to the publication of an extension to the CONSORT statement specifically for cluster randomised trials. In this paper we review the CONSORT extension to cluster randomised trials, outlining the special features of the cluster randomised trial which must be considered.

Intracluster correlation coefficients in cluster randomized trials: empirical insights into how should they be reported.

BACKGROUND: Increasingly, researchers are recognizing that there are many situations where the use of a cluster randomized trial may be more appropriate than an individually randomized trial. Similarly, the need for appropriate standards of reporting of cluster trials is more widely acknowledged. METHODS: In this paper, we describe the results of a survey to inform the appropriate reporting of the intracluster correlation coefficient (ICC)--the statistical measure of the clustering effect associated with a cluster randomized trial. RESULTS: We identified three dimensions that should be considered when reporting an ICC--a description of the dataset (including characteristics of the outcome and the intervention), information on how the ICC was calculated, and information on the precision of the ICC. CONCLUSIONS: This paper demonstrates the development of a framework for the reporting of ICCs. If adopted into routine practice, it has the potential to facilitate the interpretation of the cluster trial being reported and should help the development of new trials in the area.

Meta-analyses involving cross-over trials: methodological issues.

BACKGROUND: Meta-analysis of randomized controlled trials (RCTs) is usually based on trials where patients are randomized individually into two different, parallel, treatment groups. This paper concentrates on RCTs of a different design-two-period, two-treatment cross-over trials. METHODS: The characteristics of these trials are outlined, with detailed examples of methods for analysis for both continuous and binary data. These case studies are then extended into the context of a meta-analysis. The Cochrane Library was surveyed to assess current practice for synthesis. RESULTS: Methods are described for continuous and binary data for use both when the necessary paired data are given and also when they need to be calculated or imputed, and some suggestions are provided to help people wishing to synthesize data from cross-over trials into meta-analyses. The survey suggested that about 8% of the trials in the Cochrane library were cross-over trials and 18% of the reviews referred to such trials, although there was no consistent approach to their inclusion into the reviews. CONCLUSIONS: Methods do exist for including valuable information from two-period, two-treatment cross-over trials into quantitative reviews. However, poor reporting of cross-over trials will often impede attempts to perform a meta-analysis using the available methods.