Safety and effectiveness of recombinant human bone morphogenetic protein-2 for spinal fusion: a meta-analysis of individual-participant data.

BACKGROUND: Recombinant human bone morphogenetic protein-2 (rhBMP-2) is widely used to promote fusion in spinal surgery, but its safety has been questioned. PURPOSE: To evaluate the effectiveness and safety of rhBMP-2. DATA SOURCES: Individual-participant data obtained from the sponsor or investigators and data extracted from study publications identified by systematic bibliographic searches through June 2012. STUDY SELECTION: Randomized, controlled trials of rhBMP-2 versus iliac crest bone graft (ICBG) in spinal fusion surgery for degenerative disc disease and related conditions and observational studies in similar populations for investigation of adverse events. DATA EXTRACTION: Individual-participant data from 11 eligible of 17 provided trials sponsored by Medtronic (Minneapolis, Minnesota) (n = 1302) and 1 of 2 other eligible trials (n = 106) were included. Additional aggregate adverse event data were extracted from 35 published observational studies. DATA SYNTHESIS: Primary outcomes were pain (assessed with the Oswestry Disability Index [ODI] or Short Form-36), fusion, and adverse events. At 24 months, ODI scores were 3.5% lower (better) with rhBMP-2 than with ICBG (95% CI, 0.5% to 6.5%) and radiographic fusion was 12% higher (CI, 2% to 23%). At or shortly after surgery, pain was more common with rhBMP-2 (odds ratio, 1.78 [CI, 1.06 to 2.95]). Cancer was more common after rhBMP-2 (relative risk, 1.98 [CI, 0.86 to 4.54]), but the small number of events precluded definite conclusions. LIMITATION: The observational studies were diverse and at risk of bias. CONCLUSION: At 24 months, rhBMP-2 increases fusion rates, reduces pain by a clinically insignificant amount, and increases early postsurgical pain compared with ICBG. Evidence of increased cancer incidence is inconclusive. PRIMARY FUNDING SOURCE: Yale University Open Data Access Project.

Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods.

OBJECTIVE: Meta-analysis of individual patient data (IPD) is the "gold-standard" for synthesizing evidence across several studies. Some studies, however, may only provide aggregate data (AD). In this situation researchers might need to combine IPD with AD to utilize all the evidence available. Here, we review applied IPD meta-analysis articles to assess if and how AD is combined with IPD in practice. STUDY DESIGN AND SETTING: A systematic review of articles identified from bibliographic databases and searches. RESULTS: We identified 33 applied IPD articles that combined IPD and AD and 166 that did not. For each article, we recorded the proportion of total studies providing IPD, and found that articles combining IPD and AD had, on average, IPD available in only 64% of studies (compared to 90% in articles not combining IPD and AD). Two different methods were used to combine IPD and AD, the two-stage method and analysis of partially reconstructed IPD, but a review of methodological articles identified two further methods, multilevel modeling and Bayesian hierarchical related regression. We summarize each method to aid practitioners. CONCLUSION: Combining IPD and AD is a relevant issue for evidence synthesis, and the further development and validation of suitable meta-analysis methods is needed.

A general framework for the use of logistic regression models in meta-analysis.

Where individual participant data are available for every randomised trial in a meta-analysis of dichotomous event outcomes, "one-stage" random-effects logistic regression models have been proposed as a way to analyse these data. Such models can also be used even when individual participant data are not available and we have only summary contingency table data. One benefit of this one-stage regression model over conventional meta-analysis methods is that it maximises the correct binomial likelihood for the data and so does not require the common assumption that effect estimates are normally distributed. A second benefit of using this model is that it may be applied, with only minor modification, in a range of meta-analytic scenarios, including meta-regression, network meta-analyses and meta-analyses of diagnostic test accuracy. This single model can potentially replace the variety of often complex methods used in these areas. This paper considers, with a range of meta-analysis examples, how random-effects logistic regression models may be used in a number of different types of meta-analyses. This one-stage approach is compared with widely used meta-analysis methods including Bayesian network meta-analysis and the bivariate and hierarchical summary receiver operating characteristic (ROC) models for meta-analyses of diagnostic test accuracy.

Random-effects meta-analysis of time-to-event data using the expectation-maximisation algorithm and shrinkage estimators.

Meta-analysis of time-to-event data has proved difficult in the past because consistent summary statistics often cannot be extracted from published results. The use of individual patient data allows for the re-analysis of each study in a consistent fashion and thus makes meta-analysis of time-to-event data feasible. Time-to-event data can be analysed using proportional hazards models, but incorporating random effects into these models is not straightforward in standard software. This paper fits random-effects proportional hazards models by treating the random effects as missing data and applying the expectation-maximisation algorithm. This approach has been used before by using Markov chain Monte Carlo methods to perform the expectation step of the algorithm. In this paper, the expectation step is simplified, without sacrificing accuracy, by approximating the expected values of the random effects using simple shrinkage estimators. This provides a robust method for fitting random-effects models that can be implemented in standard statistical packages. Copyright © 2012 John Wiley & Sons, Ltd.

Reporting of industry funded study outcome data: comparison of confidential and published data on the safety and effectiveness of rhBMP-2 for spinal fusion.

OBJECTIVE: To investigate whether published results of industry funded trials of recombinant human bone morphogenetic protein 2 (rhBMP-2) in spinal fusion match underlying trial data by comparing three different data sources: individual participant data, internal industry reports, and publicly available journal publications and conference abstracts. DATA COLLECTION AND SYNTHESIS: The manufacturer of rhBMP-2 products (Medtronic; Minneapolis, MN) provided complete individual participant data and internal reports for all its studies of rhMBP-2 in spinal fusion. We identified publications and conference abstracts through comprehensive literature searches. We compared outcomes provided in the individual participant data against outcomes reported in publications. For effectiveness outcomes, we compared meta-analyses of randomised controlled trials based on each of the three data sources. For adverse events, meta-analysis of the published aggregate data was not possible and we compared the number and type of adverse events reported between data sources. RESULTS: 32 publications reported outcomes from 11 of the 17 existing manufacturer sponsored studies. For individual randomised controlled trials, 56% (9/16) to 88% (15/17) of effectiveness outcomes known to have been collected were reported in the published literature. Meta-analyses of effectiveness data were almost identical for pain outcomes and similar for fusion across the three data sources. A minority of adverse event data known to have been collected were reported in the published literature. Several journal articles reported only "serious," "related," or "unanticipated" adverse events, without defining these terms. Others reported a small proportion of the collected adverse event categories. Around 23% (533/2302) of the total adverse events collected in published randomised controlled trials have been reported in the literature, with randomised controlled trials evaluating the licensed preparation (Infuse) reporting around 11% (122/1108) of collected adverse events. CONCLUSIONS: The published literature only partially represents the total data known to have been collected on the effects of rhBMP-2. This did not lead to substantially different results for meta-analysis of effectiveness outcomes. In contrast, reporting of adverse event data in trial publications was inadequate and inconsistent to the extent that any systematic review based solely on the publicly available data would not be able to properly evaluate the safety of rhBMP-2. Analysis of individual participant data enabled the most complete, detailed, and in-depth analysis and was not more resource intensive than extracting, collating, and analysing aggregate data from multiple trial publications and conference abstracts. Confidential internal reports presented considerably more adverse event data than publications, and in the absence of individual participant data access to these reports would support more accurate and reliable investigation, with less time and effort than relying on incomplete published data.

Risk estimation versus screening performance: a comparison of six risk algorithms for cardiovascular disease.

BACKGROUND: Risk of future cardiovascular disease (CVD) events is typically estimated from risk factors such as age, sex, blood pressure and cholesterol. Many 'risk algorithms' exist to estimate CVD risk. All should have similar screening performances because of the dominant effect of age in predicting who will and will not have a CVD event, regardless of the accuracy of CVD risk estimation. Six CVD risk algorithms were compared (Framingham 1991, Framingham 2008, Reynolds risk, ASSIGN, SCORE and QRISK2), each differing in the risk factors used and in CVD outcomes. METHODS: The six algorithms were applied to a simulated sample of 500,000 people aged 40-74, based on the population of England. CVD risk was calculated for each individual using all risk algorithms, and who did and did not have a CVD event in 10 years was simulated according to those estimated risks. Screening performance was assessed by estimating the detection rate (sensitivity) and false-positive rate (1 - specificity) at a range of cut-off values of CVD risk for each algorithm. The accuracy (calibration) of risk estimation was compared across the six algorithms. RESULTS: At a 20% false-positive rate the detection rates of the six algorithms ranged from 72% to 79%. The estimated risk cut-offs to achieve the same false-positive rate varied five-fold, from 4% to 21% because of the different risk factors and outcomes considered. CONCLUSIONS: All six risk algorithms had similar screening performances. The accuracy (calibration) of CVD risk estimation does not materially affect screening performance. In distinguishing who will and will not develop CVD it is screening performance that matters rather than the accuracy of the risk estimation.

Statistical analysis of individual participant data meta-analyses: a comparison of methods and recommendations for practice.

BACKGROUND: Individual participant data (IPD) meta-analyses that obtain "raw" data from studies rather than summary data typically adopt a "two-stage" approach to analysis whereby IPD within trials generate summary measures, which are combined using standard meta-analytical methods. Recently, a range of "one-stage" approaches which combine all individual participant data in a single meta-analysis have been suggested as providing a more powerful and flexible approach. However, they are more complex to implement and require statistical support. This study uses a dataset to compare "two-stage" and "one-stage" models of varying complexity, to ascertain whether results obtained from the approaches differ in a clinically meaningful way. METHODS AND FINDINGS: We included data from 24 randomised controlled trials, evaluating antiplatelet agents, for the prevention of pre-eclampsia in pregnancy. We performed two-stage and one-stage IPD meta-analyses to estimate overall treatment effect and to explore potential treatment interactions whereby particular types of women and their babies might benefit differentially from receiving antiplatelets. Two-stage and one-stage approaches gave similar results, showing a benefit of using anti-platelets (Relative risk 0.90, 95% CI 0.84 to 0.97). Neither approach suggested that any particular type of women benefited more or less from antiplatelets. There were no material differences in results between different types of one-stage model. CONCLUSIONS: For these data, two-stage and one-stage approaches to analysis produce similar results. Although one-stage models offer a flexible environment for exploring model structure and are useful where across study patterns relating to types of participant, intervention and outcome mask similar relationships within trials, the additional insights provided by their usage may not outweigh the costs of statistical support for routine application in syntheses of randomised controlled trials. Researchers considering undertaking an IPD meta-analysis should not necessarily be deterred by a perceived need for sophisticated statistical methods when combining information from large randomised trials.

Meta-analysis of time-to-event data: a comparison of two-stage methods.

Meta-analysis is widely used to synthesise results from randomised trials. When the relevant trials collected time-to-event data, individual participant data are commonly sought from all trials. Meta-analyses of time-to-event data are typically performed using variants of the log-rank test, but modern statistical software allows for the use of maximum likelihood methods such as Cox proportional hazards models or interval-censored logistic regression. In this paper, the different approaches to the analysis of time-to-event data are examined and compared with show that log-rank test approaches are in fact first-order approximations to the maximum likelihood methods and that some methods assume proportional hazards, whereas others assume proportional odds. A simulation study is performed to compare the different methods, which shows that log-rank test approaches give biased estimates when the underlying hazard ratio or odds ratio is far from unity. It also shows that proportional hazards methods give biased results when hazards are not proportional, and proportional odds methods give biased results when odds are not proportional. Maximum likelihood models should, therefore, be preferred to log-rank test based methods for the meta-analysis of time-to-event data and any such meta-analysis should investigate whether proportional hazards or proportional odds assumptions are valid. Copyright © 2011 John Wiley & Sons, Ltd.

Meta-analysis of individual patient data from randomized trials: a review of methods used in practice.

BACKGROUND: Meta-analyses based on individual patient data (IPD) are regarded as the gold standard for systematic reviews. However, the methods used for analysing and presenting results from IPD meta-analyses have received little discussion. METHODS: We review 44 IPD meta-analyses published during the years 1999-2001. We summarize whether they obtained all the data they sought, what types of approaches were used in the analysis, including assumptions of common or random effects, and how they examined the effects of covariates. RESULTS: Twenty-four out of 44 analyses focused on time-to-event outcomes, and most analyses (28) estimated treatment effects within each trial and then combined the results assuming a common treatment effect across trials. Three analyses failed to stratify by trial, analysing the data is if they came from a single mega-trial. Only nine analyses used random effects methods. Covariate-treatment interactions were generally investigated by subgrouping patients. Seven of the meta-analyses included data from less than 80% of the randomized patients sought, but did not address the resulting potential biases. CONCLUSIONS: Although IPD meta-analyses have many advantages in assessing the effects of health care, there are several aspects that could be further developed to make fuller use of the potential of these time-consuming projects. In particular, IPD could be used to more fully investigate the influence of covariates on heterogeneity of treatment effects, both within and between trials. The impact of heterogeneity, or use of random effects, are seldom discussed. There is thus considerable scope for enhancing the methods of analysis and presentation of IPD meta-analysis.