Performance of several types of beta-binomial models in comparison to standard approaches for meta-analyses with very few studies.

BACKGROUND: Meta-analyses are used to summarise the results of several studies on a specific research question. Standard methods for meta-analyses, namely inverse variance random effects models, have unfavourable properties if only very few (2 - 4) studies are available. Therefore, alternative meta-analytic methods are needed. In the case of binary data, the "common-rho" beta-binomial model has shown good results in situations with sparse data or few studies. The major concern of this model is that it ignores the fact that each treatment arm is paired with a respective control arm from the same study. Thus, the randomisation to a study arm of a specific study is disrespected, which may lead to compromised estimates of the treatment effect. Therefore, we extended this model to a version that respects randomisation. The aim of this simulation study was to compare the "common-rho" beta-binomial model and several other beta-binomial models with standard meta-analyses models, including generalised linear mixed models and several inverse variance random effects models. METHODS: We conducted a simulation study comparing beta-binomial models and various standard meta-analysis methods. The design of the simulation aimed to consider meta-analytic situations occurring in practice. RESULTS: No method performed well in scenarios with only 2 studies in the random effects scenario. In this situation, a fixed effect model or a qualitative summary of the study results may be preferable. In scenarios with 3 or 4 studies, most methods satisfied the nominal coverage probability. The "common-rho" beta-binomial model showed the highest power under the alternative hypothesis. The beta-binomial model respecting randomisation did not improve performance. CONCLUSION: The "common-rho" beta-binomial appears to be a good option for meta-analyses of very few studies. As residual concerns about the consequences of disrespecting randomisation may still exist, we recommend a sensitivity analysis with a standard meta-analysis method that respects randomisation.

On estimands and the analysis of adverse events in the presence of varying follow-up times within the benefit assessment of therapies.

The analysis of adverse events (AEs) is a key component in the assessment of a drug's safety profile. Inappropriate analysis methods may result in misleading conclusions about a therapy's safety and consequently its benefit-risk ratio. The statistical analysis of AEs is complicated by the fact that the follow-up times can vary between the patients included in a clinical trial. This paper takes as its focus the analysis of AE data in the presence of varying follow-up times within the benefit assessment of therapeutic interventions. Instead of approaching this issue directly and solely from an analysis point of view, we first discuss what should be estimated in the context of safety data, leading to the concept of estimands. Although the current discussion on estimands is mainly related to efficacy evaluation, the concept is applicable to safety endpoints as well. Within the framework of estimands, we present statistical methods for analysing AEs with the focus being on the time to the occurrence of the first AE of a specific type. We give recommendations which estimators should be used for the estimands described. Furthermore, we state practical implications of the analysis of AEs in clinical trials and give an overview of examples across different indications. We also provide a review of current practices of health technology assessment (HTA) agencies with respect to the evaluation of safety data. Finally, we describe problems with meta-analyses of AE data and sketch possible solutions.

Methodological approach to determine minor, considerable, and major treatment effects in the early benefit assessment of new drugs.

At the beginning of 2011, the early benefit assessment of new drugs was introduced in Germany with the Act on the Reform of the Market for Medicinal Products (AMNOG). The Federal Joint Committee (G-BA) generally commissions the Institute for Quality and Efficiency in Health Care (IQWiG) with this type of assessment, which examines whether a new drug shows an added benefit (a positive patient-relevant treatment effect) over the current standard therapy. IQWiG is required to assess the extent of added benefit on the basis of a dossier submitted by the pharmaceutical company responsible. In this context, IQWiG was faced with the task of developing a transparent and plausible approach for operationalizing how to determine the extent of added benefit. In the case of an added benefit, the law specifies three main extent categories (minor, considerable, major). To restrict value judgements to a minimum in the first stage of the assessment process, an explicit and abstract operationalization was needed. The present paper is limited to the situation of binary data (analysis of 2 × 2 tables), using the relative risk as an effect measure. For the treatment effect to be classified as a minor, considerable, or major added benefit, the methodological approach stipulates that the (two-sided) 95% confidence interval of the effect must exceed a specified distance to the zero effect. In summary, we assume that our approach provides a robust, transparent, and thus predictable foundation to determine minor, considerable, and major treatment effects on binary outcomes in the early benefit assessment of new drugs in Germany. After a decision on the added benefit of a new drug by G-BA, the classification of added benefit is used to inform pricing negotiations between the umbrella organization of statutory health insurance and the pharmaceutical companies.

The Peto odds ratio viewed as a new effect measure.

Meta-analysis has generally been accepted as a fundamental tool for combining effect estimates from several studies. For binary studies with rare events, the Peto odds ratio (POR) method has become the relative effect estimator of choice. However, the POR leads to biased estimates for the OR when treatment effects are large or the group size ratio is not balanced. The aim of this work is to derive the limit of the POR estimator for increasing sample size, to investigate whether the POR limit is equal to the true OR and, if this is not the case, in which situations the POR limit is sufficiently close to the OR. It was found that the derived limit of the expected POR is not equivalent to the OR, because it depends on the group size ratio. Thus, the POR represents a different effect measure. We investigated in which situations the POR is reasonably close to the OR and found that this depends only slightly on the baseline risk within the range (0.001; 0.1) yet substantially on the group size ratio and the effect size itself. We derived the maximum effect size of the POR for different group size ratios and tolerated amounts of bias, for which the POR method results in an acceptable estimator of the OR. We conclude that the limit of the expected POR can be regarded as a new effect measure, which can be used in the presented situations as a valid estimate of the true OR.

Systematic review of surgical treatments for benign prostatic hyperplasia and presentation of an approach to investigate therapeutic equivalence (non-inferiority).

OBJECTIVE: • To assess the potential additional benefit of non-standard vs standard surgical treatments for benign prostatic hyperplasia (BPH) and to present a new methodological approach to investigate therapeutic equivalence (non-inferiority) regarding symptom reduction. PATIENTS AND METHODS: • We conducted a systematic review and searched MEDLINE, Embase and the Cochrane Library (last search: 10/2009) for randomized controlled trials (RCTs) and non-randomized controlled clinical trials (CCTs). • Eligible studies were those that included patients with symptomatic BPH requiring surgical treatment and which compared non-standard procedures (e.g. minimally invasive technologies) with standard ones (e.g. transurethral resection of the prostate, TURP). In addition, only studies analysing patient-relevant outcomes were considered (e.g. irritative and obstructive symptoms, length of hospital stay, quality of life and adverse events). • The main outcome of interest for the present analysis was superiority or non-inferiority for symptom reduction. • As no trial investigated non-inferiority, we defined a non-inferiority threshold (0.25 standard deviation) on the basis of published literature. If a non-standard procedure showed non-inferiority for symptom reduction, additional outcomes were assessed. Meta-analyses were conducted if feasible and meaningful. RESULTS: • In all, 43 mainly low-quality trials (RCTs only) compared nine non-standard surgical treatments with standard ones. • Mean follow-up ranged from 6 to 84 months. • No non-standard procedure was superior for symptom reduction. Non-inferiority for symptom reduction was shown in patients who had undergone holmium laser resection of the prostate (HoLRP) or thulium laser resection of the prostate (TmLRP). • As procedural advantages (e.g. no occurrence of transurethral resection syndrome) and other advantages (e.g. shortened hospital stay) were found, an indication of an additional benefit of HoLRP and TmLRP was determined. CONCLUSIONS: • No proof of superiority for symptom reduction has been shown for non-standard surgical treatments in patients with BPH. • There is a lack of high-quality RCTs and trials designed to investigate non-inferiority. • Future studies should define a non-inferiority threshold (ideally, uniform) a priori, so that results of individual studies are interpretable and comparable, and future systematic reviews should consider this issue.

Performing Meta-analyses with Very Few Studies.

This chapter contains a methodological framework for choosing a model for the meta-analysis of very few studies and selecting an estimation method for the chosen model by means of study characteristics and by comparing results yielded by different approaches. When the results are inconclusive between different estimation methods, it might be the best solution to refrain from a quantitative meta-analysis but to summarize the study results by means of a qualitative evidence synthesis.

Reporting of loss to follow-up information in randomised controlled trials with time-to-event outcomes: a literature survey.

BACKGROUND: To assess the reporting of loss to follow-up (LTFU) information in articles on randomised controlled trials (RCTs) with time-to-event outcomes, and to assess whether discrepancies affect the validity of study results. METHODS: Literature survey of all issues of the BMJ, Lancet, JAMA, and New England Journal of Medicine published between 2003 and 2005. Eligible articles were reports of RCTs including at least one Kaplan-Meier plot. Articles were classified as "assessable" if sufficient information was available to assess LTFU. In these articles, LTFU information was derived from Kaplan-Meier plots, extracted from the text, and compared. Articles were then classified as "consistent" or "not consistent". Sensitivity analyses were performed to assess the validity of study results. RESULTS: 319 eligible articles were identified. 187 (59%) were classified as "assessable", as they included sufficient information for evaluation; 140 of 319 (44%) presented consistent LTFU information between the Kaplan-Meier plot and text. 47 of 319 (15%) were classified as "not consistent". These 47 articles were included in sensitivity analyses. When various imputation methods were used, the results of a chi2-test applied to the corresponding 2 × 2 table changed and hence were not robust in about half of the studies. CONCLUSIONS: Less than half of the articles on RCTs using Kaplan-Meier plots provide assessable and consistent LTFU information, thus questioning the validity of the results and conclusions of many studies presenting survival analyses. Authors should improve the presentation of both Kaplan-Meier plots and LTFU information, and reviewers of study publications and journal editors should critically appraise the validity of the information provided.

Population-based screening of children for specific speech and language impairment in Germany: a systematic review.

OBJECTIVE: The study was aimed at evaluating the effectiveness of a systematic population-based screening programme for specific language impairment (SLI) in preschool children in Germany. METHODS: The study question was divided into a review of (1) evidence from studies evaluating screening programmes, (2) diagnostic instruments in the German language, and (3) studies evaluating speech and language interventions. A systematic database search was conducted between June and October 2007 and was updated in January and again in May 2008. Relevant studies were identified by 2 independent reviewers based on screened titles/abstracts and full texts. RESULTS: 4,806 studies were screened. The only existing controlled screening study did not provide data for SLI. No diagnostic study met the inclusion criteria. Sixteen randomized intervention studies were included, 3 studies contributed to the appraisal of earlier against later initiation of treatment. Most studies were of limited quality. We found indications of short-term positive effects from language therapies in children with SLI. Long-term outcomes were not investigated. No evidence supporting the advantage of earlier treatment initiation was identified. CONCLUSIONS: The benefit of population-based language screening of preschool children with SLI is not proven. Controlled screening studies are therefore necessary. For Germany, the accuracy of existing diagnostic instruments has not yet been sufficiently examined.

Ginkgo biloba in Alzheimer's disease: a systematic review.

This systematic review determines the benefit of treatment with Ginkgo biloba (Ginkgo) in Alzheimer's disease (AD) concerning patient-relevant outcomes. Bibliographic databases, clinical trial and study result registries were searched for randomized controlled trials (RCTs) in patients with AD (follow-up ≥16 weeks) comparing Ginkgo to placebo or a different treatment option. Manufacturers were asked to provide unpublished data. If feasible, data were pooled by meta-analysis. Six studies were eligible; overall, high heterogeneity was shown for most outcomes, except safety aspects. Among studies administering high-dose Ginkgo (240 mg), all studies favour treatment though effects remain heterogeneous. In this subgroup, a benefit of Ginkgo exists for activities of daily living. Cognition and accompanying psychopathological symptoms show an indication of a benefit. A harm of Ginkgo is not evident. An estimation of the effect size was not possible for any outcome. Further evidence is needed which focuses especially on subgroups of AD patients.

Long-term effects of weight-reducing interventions in hypertensive patients: systematic review and meta-analysis.

Weight loss is recommended in all major guidelines for antihypertensive therapy. We searched for randomized controlled trials investigating the effects of weight-reducing diets, pharmacologic substances, and invasive interventions for weight reduction on patient-relevant end points and blood pressure (BP) in patients with essential hypertension. No information on the effects on patient-relevant end points was available. Patients assigned to weight loss diets, orlistat, or sibutramine reduced their body weight more effectively than did patients in the usual care/placebo groups. Reduction of BP was higher in patients treated with weight loss diets (systolic BP [SBP]: weighted mean difference [WMD], -6.3 mm Hg; diastolic BP [DBP]: WMD, -3.4 mm Hg) or orlistat (SBP: WMD, -2.5 mm Hg; DBP: WMD, -2.0 mm Hg). Systolic BP increased with sibutramine treatment (WMD, 3.2 mm Hg). In patients with essential hypertension, therapy with a weight loss diet or orlistat resulted in reductions in body weight and BP. Although sibutramine treatment reduced body weight, it did not lower BP.

Methods for evidence synthesis in the case of very few studies.

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If between-study heterogeneity is expected to be present, the method of choice is a meta-analysis with random effects. The widely used DerSimonian and Laird method for meta-analyses with random effects has been criticized due to its unfavorable statistical properties, especially in the case of very few studies. A working group of the Cochrane Collaboration recommended the use of the Knapp-Hartung method for meta-analyses with random effects. However, as heterogeneity cannot be reliably estimated if only very few studies are available, the Knapp-Hartung method, while correctly accounting for the corresponding uncertainty, has very low power. Our aim is to summarize possible methods to perform meaningful evidence syntheses in the situation with only very few (ie, 2-4) studies. Some general recommendations are provided on which method should be used when. Our recommendations are based on the existing literature on methods for meta-analysis with very few studies and consensus of the authors. The recommendations are illustrated by 2 examples coming from dossier assessments of the Institute for Quality and Efficiency in Health Care.

Developing search strategies for clinical practice guidelines in SUMSearch and Google Scholar and assessing their retrieval performance.

BACKGROUND: Information overload, increasing time constraints, and inappropriate search strategies complicate the detection of clinical practice guidelines (CPGs). The aim of this study was to provide clinicians with recommendations for search strategies to efficiently identify relevant CPGs in SUMSearch and Google Scholar. METHODS: We compared the retrieval efficiency (retrieval performance) of search strategies to identify CPGs in SUMSearch and Google Scholar. For this purpose, a two-term GLAD (GuideLine And Disease) strategy was developed, combining a defined CPG term with a specific disease term (MeSH term). We used three different CPG terms and nine MeSH terms for nine selected diseases to identify the most efficient GLAD strategy for each search engine. The retrievals for the nine diseases were pooled. To compare GLAD strategies, we used a manual review of all retrievals as a reference standard. The CPGs detected had to fulfil predefined criteria, e.g., the inclusion of therapeutic recommendations. Retrieval performance was evaluated by calculating so-called diagnostic parameters (sensitivity, specificity, and "Number Needed to Read" [NNR]) for search strategies. RESULTS: The search yielded a total of 2830 retrievals; 987 (34.9%) in Google Scholar and 1843 (65.1%) in SUMSearch. Altogether, we found 119 unique and relevant guidelines for nine diseases (reference standard). Overall, the GLAD strategies showed a better retrieval performance in SUMSearch than in Google Scholar. The performance pattern between search engines was similar: search strategies including the term "guideline" yielded the highest sensitivity (SUMSearch: 81.5%; Google Scholar: 31.9%), and search strategies including the term "practice guideline" yielded the highest specificity (SUMSearch: 89.5%; Google Scholar: 95.7%), and the lowest NNR (SUMSearch: 7.0; Google Scholar: 9.3). CONCLUSION: SUMSearch is a useful tool to swiftly gain an overview of available CPGs. Its retrieval performance is superior to that of Google Scholar, where a search is more time consuming, as substantially more retrievals have to be reviewed to detect one relevant CPG. In both search engines, the CPG term "guideline" should be used to obtain a comprehensive overview of CPGs, and the term "practice guideline" should be used if a less time consuming approach for the detection of CPGs is desired.

Efficacy of the NMDA-receptor antagonist memantine in patients with chronic phantom limb pain--results of a randomized double-blinded, placebo-controlled trial.

Phantom limb pain (PLP) associated neuroplastic changes are partly mediated by excitatory amino acids at NMDA receptor sites. This study was undertaken to deduce if NMDA-receptor antagonists may be effective in patients with chronic PLP. Therefore a four week double-blinded, randomized placebo-controlled trial was performed to evaluate the efficacy of 30 mg memantine/day, an orally administrable NMDA receptor antagonist.Thirty-six patients, 18 per group, with a history of at least 12 months PLP and an average pain of at least 4 on the 11-point numeric rating scale (NRS) were enrolled. The patients completed a standardized questionnaire before the trial. PLP intensity and the level of eight complaints were assessed during the trial. Number needed to treat (NNT) was calculated based on the average PLP during the 3rd week (steady state). In both groups, PLP declined significantly in comparison with the baseline (verum: 5.1 (+/-2.1) to 3,8 (+/-2,3), placebo from 5.1 (+/-2.0) to 3.2 (+/-1,46) NRS) without a re-rising of the PLP during the washout period. Mean pain relief was 47% in the memantine group (10 patients reported more than 50% relief), 40% in the placebo group (6>50%): NNT were 4.5 (KI: 2.1-10.6). Analysis of covariance demonstrated a significant impact only on the prior PLP intensity, but no treatment effect. Two patients have demonstrated long-term pain relief under memantine until now (16 months). The total number of slight adverse events were comparable in both groups, but the overall number of severe events was higher in the memantine group (P<0.05). This trial failed to demonstrate a significant clinical benefit of the NMDA-receptor antagonist memantine in chronic PLP. The administration of a higher dosage is probably not tolerable.

Correlation of age with in vivo expression of endothelial markers.

The importance of endothelial senescence as a pathogenetic factor in age-related vascular alterations has almost exclusively been studied in vitro. However, the in vitro-findings have rarely been compared with histomorphological changes in aging human tissue or in age-related degenerative diseases. Therefore, we compared the expression of the endothelial marker CD34 and the procoagulant protein von Willebrand factor (vWf) in lung endothelium by conventional immunohistochemistry and confocal laser scan microscopy (CLSM), taking the age of the patients into consideration. The staining reactions were statistically analysed by covariance analysis. With age the endothelial staining intensity of CD34 increased in arteries and veins, but decreased in arterioles, capillaries and venules. For vWf, on the contrary, the endothelial staining intensity increased with age in all types of vessels. CLSM confirmed a mosaic staining pattern. This study demonstrates age-associated phenotypical alterations of CD34 and vWf. Whether the down-regulation of CD34 correlates with an age-associated reduction of the angiogenetic properties of EC or an age-related over-expression of vWf as a relevant cofactor for the raised coagulatory activity and the increase in thrombotic diseases resp coronary heart disease in older patients, remains subject to debate.

A note on the graphical presentation of prediction intervals in random-effects meta-analyses.

BACKGROUND: Meta-analysis is used to combine the results of several related studies. Two different models are generally applied: the fixed-effect (FE) and random-effects (RE) models. Although the two approaches estimate different parameters (that is, the true effect versus the expected value of the distribution of true effects) in practice, the graphical presentation of results is the same for both models. This means that in forest plots of RE meta-analyses, no estimate of the between-study variation is usually given graphically, even though it provides important information about the heterogeneity between the study effect sizes. FINDINGS: In addition to the point estimate of the between-study variation, a prediction interval (PI) can be used to determine the degree of heterogeneity, as it provides a region in which about 95% of the true study effects are expected to be found. To distinguish between the confidence interval (CI) for the average effect and the PI, it may also be helpful to include the latter interval in forest plots. We propose a new graphical presentation of the PI; in our method, the summary statistics in forest plots of RE meta-analyses include an additional row, '95% prediction interval', and the PI itself is presented in the form of a rectangle below the usual diamond illustrating the estimated average effect and its CI. We then compare this new graphical presentation of PIs with previous proposals by other authors. The way the PI is presented in forest plots is crucial. In previous proposals, the distinction between the CI and the PI has not been made clear, as both intervals have been illustrated either by a diamond or by extra lines added to the diamond, which may result in misinterpretation. CONCLUSIONS: To distinguish graphically between the results of an FE and those of an RE meta-analysis, it is helpful to extend forest plots of the latter approach by including the PI. Clear presentation of the PI is necessary to avoid confusion with the CI of the average effect estimate.

Expression of CD34 in pulmonary endothelial cells in vivo.

OBJECTIVES: The morphological phenotype of endothelial cells (EC) is specific for individual types of vessels. There are, however, no studies on the phenotypical diversity of EC in human lung tissue. The influences exerted by physiological factors (e.g. age, sex) or pathological factors (e.g. pulmonary hypertension) on EC have not been ascertained up to now. METHODS: In order to determine the influence of pulmonary hypertension (PH), age and sex on EC, we localized the expression of the endothelial marker CD34 immunohistologically by light microscopy and laser scanning microscopy in lung tissue specimens from children and adults. RESULTS: Capillary EC showed a stronger staining reaction than EC in arteries, veins, arterioles or venules. The staining intensity increased with age in veins and arteries and decreased with age in venules, arterioles and capillaries. Sex exerted no statistically significant influence. CD34 was generally more strongly expressed in specimens with PH than in those without. CONCLUSION: The study demonstrates for the first time that (1) CD34 is heterogeneously expressed by human pulmonary EC, and (2) the physiological/pathophysiological factors age/PH influence CD34 expression. Hence a correlation between CD34 expression and its role as adhesion molecule and a link between CD34 expression and maturation are subject of discussion.

Expression of von Willebrand factor by human pulmonary endothelial cells in vivo.

BACKGROUND AND OBJECTIVES: Whether von Willebrand factor (vWf) is variably expressed by endothelial cells (EC) of the human vascular tree or not is not known. Studies on animals showed that the varying degrees of vWf expression in pulmonary vessels was a reflection of EC heterogeneity. Neither the influence of age or sex nor that of pathophysiological factors such as pulmonary hypertension (PH) on vWf expression has been systematically analysed up to now. However, such information is essential for the design and delivery of site-selective drugs and genes as well as for the analysis of EC culture systems. METHODS: The variable degrees of vWf expression in lung tissue specimens from 64 patients (age: 6 weeks to 86 years) with and without PH were studied immunohistochemically and analysed statistically. RESULTS: CD31-specific antibody was used as a control stain for EC. It produced equally strong staining reactions in all pulmonary EC. In contrast, vWf-specific antibody yielded negative or weakly positive staining reactions in capillary EC. The staining intensities increased hand in hand with vessel calibres. Besides, they increased statistically significantly with age and PH. Sex had no influence on vWf expression. CONCLUSION: These findings show that (1) vWf expression by pulmonary EC is heterogeneous and specific for individual types of vessels, (2) age and PH enhance vWf expression, (3) vWf expression is indicative of altered EC activation but not of EC defects, and (4) elevated plasma levels of vWf correlate positively with raised coagulatory activities in older patients and in patients with PH.