Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies.

BACKGROUND: The design of randomised controlled trials (RCTs) should incorporate characteristics (such as concealment of randomised allocation and blinding of participants and personnel) that avoid biases resulting from lack of comparability of the intervention and control groups. Empirical evidence suggests that the absence of such characteristics leads to biased intervention effect estimates, but the findings of different studies are not consistent. OBJECTIVES: To examine the influence of unclear or inadequate random sequence generation and allocation concealment, and unclear or absent double blinding, on intervention effect estimates and between-trial heterogeneity, and whether or not these influences vary with type of clinical area, intervention, comparison and outcome measure. DATA SOURCES AND METHODS: Data were combined from seven contributing meta-epidemiological studies (collections of meta-analyses in which trial characteristics are assessed and results recorded). The resulting database was used to identify and remove overlapping meta-analyses. Outcomes were coded such that odds ratios < 1 correspond to beneficial intervention effects. Outcome measures were classified as mortality, other objective or subjective. We examined agreement between assessments of trial characteristics in trials assessed in more than one contributing study. We used hierarchical Bayesian bias models to estimate the effect of trial characteristics on average bias [quantified as ratios of odds ratios (RORs) with 95% credible intervals (CrIs) comparing trials with and without a characteristic] and in increasing between-trial heterogeneity. RESULTS: The analysis data set contained 1973 trials included in 234 meta-analyses. Median kappa statistics for agreement between assessments of trial characteristics were: sequence generation 0.60, allocation concealment 0.58 and blinding 0.87. Intervention effect estimates were exaggerated by an average 11% in trials with inadequate or unclear (compared with adequate) sequence generation (ROR 0.89, 95% CrI 0.82 to 0.96); between-trial heterogeneity was higher among such trials. Bias associated with inadequate or unclear sequence generation was greatest for subjective outcomes (ROR 0.83, 95% CrI 0.74 to 0.94) and the increase in heterogeneity was greatest for such outcomes [standard deviation (SD) 0.20, 95% CrI 0.03 to 0.32]. The effect of inadequate or unclear (compared with adequate) allocation concealment was greatest among meta-analyses with a subjectively assessed outcome intervention effect (ROR 0.85, 95% CrI 0.75 to 0.95), and the increase in between-trial heterogeneity was also greatest for such outcomes (SD 0.20, 95% CrI 0.02 to 0.33). Lack of, or unclear, double blinding (compared with double blinding) was associated with an average 13% exaggeration of intervention effects (ROR 0.87, 95% CrI 0.79 to 0.96), and between-trial heterogeneity was increased for such studies (SD 0.14, 95% CrI 0.02 to 0.30). Average bias (ROR 0.78, 95% CrI 0.65 to 0.92) and between-trial heterogeneity (SD 0.37, 95% CrI 0.19 to 0.53) were greatest for meta-analyses assessing subjective outcomes. Among meta-analyses with subjectively assessed outcomes, the effect of lack of blinding appeared greater than the effect of inadequate or unclear sequence generation or allocation concealment. CONCLUSIONS: Bias associated with specific reported study design characteristics leads to exaggeration of beneficial intervention effect estimates and increases in between-trial heterogeneity. For each of the three characteristics assessed, these effects were greatest for subjectively assessed outcomes. Assessments of the risk of bias in RCTs should account for these findings. Further research is needed to understand the effects of attrition bias, as well as the relative importance of blinding of patients, care-givers and outcome assessors, and thus separate the effects of performance and detection bias. FUNDING: National Institute for Health Research Health Technology Assessment programme.

Blinded trials taken to the test: an analysis of randomized clinical trials that report tests for the success of blinding.

BACKGROUND: Blinding can reduce bias in randomized clinical trials, but blinding procedures may be unsuccessful. Our aim was to assess how often randomized clinical trials test the success of blinding, the methods involved and how often blinding is reported as being successful. METHODS: We analysed a random sample of blinded randomized clinical trials indexed in the The Cochrane Central Register of Controlled Trials and published in 2001. We identified 1599 blinded trials, and noted if they had conducted any test for the success of blinding. We also selected 200 trials randomly that did not report any such test, and sent a questionnaire to the corresponding authors asking them if they had conducted any tests. RESULTS: Thirty-one out of 1599 trials (2%) reported tests for the success of blinding. Test methods varied, and reporting was generally incomplete. Blinding was considered successful in 14 out of the 31 trials (45%) and unclear in 10 (32%). Of the seven trials (23%) reporting unsuccessful blinding the risk of a biased trial result was either not addressed or was discounted in six cases. We received 130 questionnaires from trial authors (65%) of which 15 (12%) informed that they had conducted, but not published, tests. CONCLUSIONS: Blinding is rarely tested. Test methods vary, and the reporting of tests, and test results, is incomplete. There is a considerable methodological uncertainty how best to assess blinding, and an urgent need for improved methodology and improved reporting.

Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials.

Methodological deficiencies are known to affect the results of randomized trials. There are several components of trial quality, which, when inadequately attended to, may bias the treatment effect under study. The extent of this bias, so far only vaguely known, is currently being investigated by 'meta-epidemiological' re-analysis of data collected as part of systematic reviews. As inadequate quality components often co-occur, we maintain that the suspected biases must be evaluated simultaneously. Furthermore, the biases cannot safely be assumed to be homogeneous across systematic reviews. Therefore, a stable multivariable method that allows for heterogeneity is needed for assessing the 'bias coefficients'. We present two general statistical models for analysis of a study of 523 randomized trials from 48 meta-analyses in a random sample of Cochrane reviews: a logistic regression model uses the design of the trials as such to give estimates; a weighted regression model incorporates between-trial variation and thus gives wider confidence intervals, but is computationally lighter and can be used with trials of more general design. In both models, heterogeneity in the bias coefficients can be incorporated in two ways. A stratification approach pools the estimates from models estimated on subgroups of the data. We explore stratification by reviews and by broad trial types, the latter of which gives larger subgroups of the data, circumventing instabilities. A multilevel approach also avoids instabilities and addresses the more fundamental problem of interpretation of the pooled multivariable effect in the presence of heterogeneity.

Dopaminergic agonists for hepatic encephalopathy.

BACKGROUND: Hepatic encephalopathy may be associated with an impairment of the dopaminergic neurotransmission. Dopaminergic agonists may therefore have a beneficial effect on patients with hepatic encephalopathy. OBJECTIVES: To evaluate the beneficial and harmful effects of dopaminergic agonists for patients with hepatic encephalopathy. SEARCH STRATEGY: Trials were identified through The Cochrane Hepato-Biliary Group Controlled Trials Register (July 2004), The Cochrane Central Register of Controlled Trials (Issue 3, 2004), MEDLINE (1966-2004/07), EMBASE (1980-2004/07), manual searches of bibliographies and journals, authors of trials, and pharmaceutical companies. SELECTION CRITERIA: All randomised trials comparing dopaminergic agonists versus placebo or no intervention for hepatic encephalopathy. DATA COLLECTION AND ANALYSIS: Trial inclusion and data extraction were made independently by two reviewers. Binary outcomes are reported as odds ratios (OR) with 95% confidence intervals (CI) based on a random effects model. The presence of statistical heterogeneity was explored by the chi-squared test with significance set at P < 0.1. Potential sources of heterogeneity were explored through subgroup analyses with regard to the type of hepatic encephalopathy and type of dopaminergic agonist. MAIN RESULTS: Five trials were included. Four trials had low methodological quality. Compared with placebo or no treatment, dopaminergic agonists had no significant effect on the risk of no improvement (OR 0.33, 95% CI 0.01 to 11.25, two trials, 80 patients) or mortality (OR 1.11, 95% CI 0.34 to 3.54, four trials, 139 patients). There was significant heterogeneity (P = 0.09) among trial results on the risk of no improvement, but not on mortality (P = 0.19). The treatment response was not significantly different with regard to the type of hepatic encephalopathy or dopaminergic agonist, but the analyses had very low power to detect potential differences. There was a nonsignificant trend that dopaminergic agonists may be associated with adverse events (OR 8.33, 95% CI 0.37 to 187.74, 2 trials, 13 patients). All adverse events (n = 7) occurred in the experimental group. REVIEWERS' CONCLUSIONS: This review does not provide evidence that dopaminergic agonists are of benefit to patients with acute or chronic hepatic encephalopathy, or fulminant hepatic failure. The review is limited by the small number of trials performed within this field, the low number of patients randomised in each trial, and the low methodological quality of included trials. Accordingly, there is also insufficient evidence to exclude a potential beneficial effect. Dopaminergic agonists should not be used for hepatic encephalopathy, but may be assessed in future randomised clinical trials.

Benzodiazepine receptor antagonists for hepatic encephalopathy.

BACKGROUND: Hepatic encephalopathy may be associated with accumulation of substances that bind to a receptor-complex in the brain resulting in neural inhibition. Benzodiazepine receptor antagonists may have a beneficial effect on patients with hepatic encephalopathy. OBJECTIVES: To evaluate the beneficial and harmful effects of benzodiazepine receptor antagonists for patients with hepatic encephalopathy. SEARCH STRATEGY: Eligible trials were identified through The Cochrane Hepato-Biliary Group Controlled Trials Register, The Cochrane Controlled Trials Register on The Cochrane Library, MEDLINE and EMBASE (last search: January 2004), reference lists of relevant articles, authors of trials, and pharmaceutical companies. SELECTION CRITERIA: Randomised trials comparing any benzodiazepine receptor antagonist versus placebo or no intervention for hepatic encephalopathy. DATA COLLECTION AND ANALYSIS: Two reviewers independently included trials and extracted data. Binary outcomes are reported as risk difference (RD) with 95% confidence intervals (CI) based on a random effects model. Statistical heterogeneity was explored by a chi-squared test with significance set at P < 0.1. The inconsistency across trials was assessed by I(2). Potential sources of heterogeneity were explored through subgroup analyses. MAIN RESULTS: Thirteen randomised trials with 805 patients were included. Eight trials used a crossover design. All trials were double-blind and assessed flumazenil versus placebo. Data on all outcomes could not be extracted from all trials. The included patients had a favourable prognosis (361/390 [93%] survived in the flumazenil group versus 345/376 [92%] in the placebo group). Flumazenil had a significant beneficial effect on improvement of hepatic encephalopathy at the end of treatment (RD 0.28; 95% CI 0.20 to 0.37, eight trials). Flumazenil had no significant effect on recovery (RD 0.13; 95% CI -0.09 to 0.36, two trials) or mortality RD 0.01; 95% CI -0.05 to 0.07, 10 trials). Flumazenil may be associated with adverse events, but trial results were heterogeneous. REVIEWERS' CONCLUSIONS: Flumazenil had a significant beneficial effect on short-term improvement of hepatic encephalopathy in patients with cirrhosis and a highly favourable prognosis. Flumazenil had no significant effect on recovery or survival. Considering the fluctuating nature of hepatic encephalopathy, future trials should use a parallel design and assess if treatment with flumazenil leads to a sustained improvement or increased recovery and survival. Until this has been demonstrated, flumazenil may be considered for patients with chronic liver disease and hepatic encephalopathy, but cannot be recommended for routine clinical use.

Nonabsorbable disaccharides for hepatic encephalopathy.

BACKGROUND: Nonabsorbable disaccharides (lactulose or lactitol) are considered the treatment of choice for hepatic encephalopathy. OBJECTIVES: To assess the beneficial and harmful effects of nonabsorbable disaccharides for patients with hepatic encephalopathy. SEARCH STRATEGY: Trials were identified through The Cochrane Hepato-Biliary Group Controlled Trials Register (March 2003), The Cochrane Central Register of Controlled Trials (Issue 1, 2003), MEDLINE (1966 to 2003/03), EMBASE (1980 to 2003/03), manual searches of bibliographies and journals, authors of trials, and pharmaceutical companies. SELECTION CRITERIA: Randomised trials comparing lactulose or lactitol versus no intervention, placebo, or antibiotics and trials comparing lactulose versus lactitol for hepatic encephalopathy. DATA COLLECTION AND ANALYSIS: The primary outcome measures included no improvement of hepatic encephalopathy and all-cause mortality. Binary outcomes are reported as relative risks (RR) based on a random effects model. Subgroup analyses were performed with regard to methodological quality and form of hepatic encephalopathy. MAIN RESULTS: Thirty trials assessed nonabsorbable disaccharides versus placebo, no intervention, or antibiotics or assessed lactulose versus lactitol. We could not extract data from all trials. Compared with placebo or no intervention, nonabsorbable disaccharides had no statistically significant effect on mortality (RR 0.41, 95% CI 0.02 to 8.68, four trials), but appeared to reduce the risk of no improvement of hepatic encephalopathy (RR 0.62, 95% CI 0.46 to 0.84, six trials). However, this result may reflect bias due to low methodological quality of the majority of included trials. Trials of high methodological quality found no significant effect of nonabsorbable disaccharides on the risk of no improvement (RR 0.92, 95% CI 0.42 to 2.04, two trials). We found no statistically significant difference between lactulose and lactitol on mortality (two trials) or risk of no improvement (four trials). However, our meta-analyses were underpowered to establish whether these treatments have comparable effect. Nonabsorbable disaccharides appeared to be inferior to antibiotics on reducing the risk of no improvement (RR 1.24, 95% CI 1.02 to 1.50, 10 trials). REVIEWERS' CONCLUSIONS: This systematic review questions the beneficial effects of nonabsorbable disaccharides and highlights that there is insufficient high-quality evidence to support this treatment. We found that antibiotics appeared to be superior to nonabsorbable disaccharides in improving hepatic encephalopathy, but it is unclear whether this difference in treatment effect is clinically important to patients. Nonabsorbable disaccharides should not serve as comparator in randomised trials on hepatic encephalopathy.

Artificial and bioartificial support systems for liver failure.

BACKGROUND: Artificial and bioartificial liver support systems may 'bridge' patients with acute or acute-on-chronic liver failure to liver transplantation or recovery. OBJECTIVES: To evaluate beneficial and harmful effects of artificial and bioartificial support systems for acute and acute-on-chronic liver failure. SEARCH STRATEGY: Trials were identified through The Cochrane Hepato-Biliary Group Controlled Trials Register (September 2002), The Cochrane Central Register of Controlled Trials on The Cochrane Library (Issue 3, 2002), MEDLINE (1966 - September 2002), EMBASE (1985 - September 2002), and The Chinese Biomedical Database (September 2002), manual searches of bibliographies and journals, authors of trials, and pharmaceutical companies. SELECTION CRITERIA: Randomised clinical trials on artificial or bioartificial support systems for acute or acute on-chronic liver failure were included irrespective of blinding, publication status, or language. Non-randomised studies were included in explorative analyses. DATA COLLECTION AND ANALYSIS: Data were extracted independently by three reviewers. Results were presented as relative risks (RR) with 95% confidence intervals (CI). Sources of heterogeneity were explored through sensitivity analyses and meta-regression. The primary outcome was mortality. MAIN RESULTS: Twelve trials on artificial or bioartificial support systems versus standard medical therapy (483 patients) and two trials comparing different artificial support systems (105 patients) were included. Most trials had unclear methodological quality. Compared to standard medical therapy, support systems had no significant effect on mortality (RR 0.86; 95% CI 0.65-1.12) or bridging to liver transplantation (RR 0.87; 95% CI 0.73-1.05), but a significant beneficial effect on hepatic encephalopathy (RR 0.67; 95% CI 0.52-0.86). Meta-regression indicated that the effect of support systems depended on the type of liver failure (P = 0.03). In subgroup analyses, artificial support systems appeared to reduce mortality by 33% in acute-on-chronic liver failure (RR 0.67; 95% CI 0.51-0.90), but not in acute liver failure (RR 0.95; 95% CI 0.71-1.29). Two trials comparing artificial support systems showed significant mortality reductions with intermittent versus continuous haemofiltration (RR 0.58; 95% CI 0.36-0.94) and no significant difference between five versus ten hours of charcoal haemoperfusion (RR 1.03; 95% CI 0.65-1.62). The incidence of adverse events was inconsistently reported. REVIEWER'S CONCLUSIONS: This Review indicates that artificial support systems may reduce mortality in acute-on-chronic liver failure. Artificial and bioartificial support systems did not appear to affect mortality in acute liver failure. However, considering the strength of the evidence additional randomised clinical trials are needed before any support system can be recommended for routine use.

Branched-chain amino acids for hepatic encephalopathy.

BACKGROUND: Hepatic encephalopathy may be caused by a decreased plasma ratio of branched-chain amino acids (BCAA) to aromatic amino acids. Treatment with BCAA may therefore have a beneficial effect on patients with hepatic encephalopathy. OBJECTIVES: To evaluate the beneficial and harmful effects of BCAA for patients with hepatic encephalopathy. SEARCH STRATEGY: We identified trials through The Cochrane Hepato-Biliary Group Controlled Trials Register (September 2002), (Issue 3, 2002), MEDLINE (1966-2002/09) and EMBASE (1980-2002/05), manual searches of bibliographies and journals, authors of trials, and pharmaceutical companies. SELECTION CRITERIA: Randomised trials comparing BCAA with any kind of control therapy for hepatic encephalopathy were included, regardless of blinding, language, or publication status. DATA COLLECTION AND ANALYSIS: Trial inclusion and data extraction were made independently by two reviewers. Our primary outcome was improvement of hepatic encephalopathy. Statistical heterogeneity was tested using random effects and fixed effect models. Binary outcomes are reported as risk ratios (RR) based on a random effects model. MAIN RESULTS: Eleven randomised trials (556 patients) assessing BCAA versus carbohydrates, neomycin/lactulose, or isonitrogenous control were included. The median number of patients in each trial was 55 (range 22 to 75). Follow-up after treatment was reported in four trials (median 17 days (range 6 to 30 days)). Compared to the control regimens, BCAA significantly increased the number of patients improving from hepatic encephalopathy at the end of treatment (risk ratio (RR) 1.31, 95% confidence interval (CI) 1.04 to 1.66, nine trials). We found no evidence of an effect of BCAA on survival (RR 1.06, 95% CI 0.98 to 1.14, eight trials) or adverse events (RR 0.97, 95% CI 0.41 to 2.31, three trials). Sensitivity analyses indicated that methodological quality had significant impact on the results. We found no evidence of an effect of BCAA on improvement of hepatic encephalopathy in trials with adequate generation of the allocation sequence (RR 1.01, 95% CI 0.84 to 1.23, three trials), adequate allocation concealment (RR 1.09, 95% CI 0.89 to 1.33, five trials), or adequate double-blinding (RR 1.20, 95% CI 0.83 to 1.73, three trials). REVIEWER'S CONCLUSIONS: We did not find convincing evidence that BCAA had a significant beneficial effect on patients with hepatic encephalopathy. The trials performed in this field were small with short follow-up and most had low methodological quality.

Benzodiazepine receptor antagonists for acute and chronic hepatic encephalopathy.

BACKGROUND: The pathogenesis of hepatic encephalopathy is unknown. It has been suggested that liver failure leads to the accumulation of substances that bind to a receptor-complex in the brain resulting in neural inhibition which may progress to coma. Several trials have assessed benzodiazepine receptor antagonists for hepatic encephalopathy, but the results are conflicting. OBJECTIVES: To evaluate the efficacy and safety of benzodiazepine receptor antagonists for patients with acute or chronic hepatic encephalopathy. SEARCH STRATEGY: Eligible trials were identified through The Cochrane Hepato-Biliary Group Controlled Trials Register, The Cochrane Controlled Trials Register, MEDLINE, EMBASE, reference lists of relevant articles, authors of trials, and the pharmaceutical company known to produce benzodiazepine receptor antagonists. SELECTION CRITERIA: Randomised trials comparing any benzodiazepine receptor antagonist versus placebo or no intervention for hepatic encephalopathy were included, regardless of language or publication status. DATA COLLECTION AND ANALYSIS: Trial inclusion and data extraction were made independently by two contributors. Depending on the presence or absence of significant heterogeneity (P<0.1) a random or fixed effect model was used. Potential causes for heterogeneity were explored by sensitivity analyses. MAIN RESULTS: Twelve randomised trials with 765 patients were included. Eight trials used a crossover design. All trials were double-blind and assessed flumazenil versus placebo. Data on all outcomes could not be extracted from all trials. The included patients had a favourable prognosis (341/370 (92%) survived in the flumazenil group versus 325/356 (91%) in the placebo group). Flumazenil had no significant effect on full recovery (two trials), survival (nine trials), or on the occurrence of adverse events (five trials). However, flumazenil was associated with a significant effect on improvement of hepatic encephalopathy compared to placebo at the end of treatment (103/346 (30%) versus 23/332 (7 %), risk difference 0.23, 95% confidence interval 0.18 to 0.28, five trials). REVIEWER'S CONCLUSIONS: Flumazenil had no significant effect on recovery or survival from hepatic encephalopathy. However, flumazenil had a significant effect on short-term improvement of hepatic encephalopathy in some patients with chronic liver disease and a highly favourable prognosis. Considering the fluctuating nature of hepatic encephalopathy, future trials should use a parallel design and assess if treatment with flumazenil leads to a sustained improvement or increased recovery and survival. Until this has been demonstrated, flumazenil may be considered for patients with chronic liver disease and hepatic encephalopathy, but cannot be recommended for routine clinical use.

Time interval from human chorionic gonadotrophin (HCG) injection to follicular rupture.

The aim of this study was to estimate the time interval from human chorionic gonadotrophin (HCG) injection to follicular rupture. Furthermore, it was observed whether there was any effect on the pregnancy rate if the insemination was performed at the time of follicular rupture. In a programme of intrauterine insemination 37 consecutive cycles in 32 patients were monitored after stimulation with clomiphene citrate. HCG was administered by i.m. injection when a leading follicle of at least 18 mm in diameter was observed sonographically. All patients were monitored by sonography with 1 h intervals from 32 h after HCG injection until the first rupture of a follicle. Insemination was performed immediately after the first follicular rupture. The pregnancy rate was 16% (5/32). In 66%, the largest follicle was the first to rupture. The mean time interval from HCG administration to first follicular rupture was 38.3 h (SEM = 0.54; range = 34-46). Our findings support the concept that ovulation occurs about 38 h after HCG administration. The pregnancy rate was within the normal range, although insemination was performed at the time of follicular rupture. The largest follicle was not always the first to rupture.