Potential impact of missing outcome data on treatment effects in systematic reviews: imputation study.

OBJECTIVE: To assess the risk of bias associated with missing outcome data in systematic reviews. DESIGN: Imputation study. SETTING: Systematic reviews. POPULATION: 100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome. MAIN OUTCOME MEASURES: Median percentage change in the relative effect estimate when applying each of the following assumption (four commonly discussed but implausible assumptions (best case scenario, none had the event, all had the event, and worst case scenario) and four plausible assumptions for missing data based on the informative missingness odds ratio (IMOR) approach (IMOR 1.5 (least stringent), IMOR 2, IMOR 3, IMOR 5 (most stringent)); percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. Sensitivity analyses based on the eight different methods of handling missing data were conducted. RESULTS: 100 systematic reviews with 653 randomised controlled trials were included. When applying the implausible but commonly discussed assumptions, the median change in the relative effect estimate varied from 0% to 30.4%. The percentage of meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario), and 26% changed direction with the worst case scenario. When applying the plausible assumptions, the median percentage change in relative effect estimate varied from 1.4% to 7.0%. The percentage of meta-analyses crossing the threshold of the null effect varied from 6% (IMOR 1.5) to 22% (IMOR 5) of meta-analyses, and 2% changed direction with the most stringent (IMOR 5). CONCLUSION: Even when applying plausible assumptions to the outcomes of participants with definite missing data, the average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. Systematic review authors should present the potential impact of missing outcome data on their effect estimates and use this to inform their overall GRADE (grading of recommendations assessment, development, and evaluation) ratings of risk of bias and their interpretation of the results.

Meta-Analyses Proved Inconsistent in How Missing Data Were Handled Across Their Included Primary Trials: A Methodological Survey.

BACKGROUND: How systematic review authors address missing data among eligible primary studies remains uncertain. OBJECTIVE: To assess whether systematic review authors are consistent in the way they handle missing data, both across trials included in the same meta-analysis, and with their reported methods. METHODS: We first identified 100 eligible systematic reviews that included a statistically significant meta-analysis of a patient-important dichotomous efficacy outcome. Then, we successfully retrieved 638 of the 653 trials included in these systematic reviews' meta-analyses. From each trial report, we extracted statistical data used in the analysis of the outcome of interest to compare with the data used in the meta-analysis. First, we used these comparisons to classify the "analytical method actually used" for handling missing data by the systematic review authors for each included trial. Second, we assessed whether systematic reviews explicitly reported their analytical method of handling missing data. Third, we calculated the proportion of systematic reviews that were consistent in their "analytical method actually used" across trials included in the same meta-analysis. Fourth, among systematic reviews that were consistent in the "analytical method actually used" across trials and explicitly reported on a method for handling missing data, we assessed whether the "analytical method actually used" and the reported methods were consistent. RESULTS: We were unable to determine the "analytical method reviews actually used" for handling missing outcome data among 397 trials. Among the remaining 241, systematic review authors most commonly conducted "complete case analysis" (n=128, 53%) or assumed "none of the participants with missing data had the event of interest" (n=58, 24%). Only eight of 100 systematic reviews were consistent in their approach to handling missing data across included trials, but none of these reported methods for handling missing data. Among seven reviews that did explicitly report their analytical method of handling missing data, only one was consistent in their approach across included trials (using complete case analysis), and their approach was inconsistent with their reported methods (assumed all participants with missing data had the event). CONCLUSION: The majority of systematic review authors were inconsistent in their approach towards reporting and handling missing outcome data across eligible primary trials, and most did not explicitly report their methods to handle missing data. Systematic review authors should clearly identify missing outcome data among their eligible trials, specify an approach for handling missing data in their analyses, and apply their approach consistently across all primary trials.

Potentially missing data are considerably more frequent than definitely missing data: a methodological survey of 638 randomized controlled trials.

BACKGROUND AND OBJECTIVE: Missing data for the outcomes of participants in randomized controlled trials (RCTs) are a key element of risk of bias assessment. However, it is not always clear from RCT reports whether some categories of participants were followed-up or not (i.e., do or do not have missing data) nor how the RCT authors dealt with missing data in their analyses. Our objectives were to describe how RCT authors (1) report on different categories of participants that might have missing data, (2) handle these categories in the analysis, and (3) judge the risk of bias associated with missing data. METHODS: We surveyed all RCT reports included in 100 clinical intervention systematic reviews (SRs), half of which were Cochrane SRs. Eligible SRs reported a group-level meta-analysis of a patient-important dichotomous efficacy outcome, with a statistically significant effect estimate. Eleven reviewers, working in pairs, independently extracted data from the primary RCT reports included in the SRs. We predefined 19 categories of participants that might have missing data. Then, we classified these participants as follows: "explicitly followed-up," "explicitly not followed-up" (i.e., definitely missing data), or "unclear follow-up status" (i.e., potentially missing data). RESULTS: Of 638 eligible RCTs, 400 (63%) reported on at least one of the predefined categories of participants that might have missing data. The median percentage of participants who were explicitly not followed-up was 5.8% (interquartile range 2.2-14.8%); it was 9.7% (4.1-14.9%) for participants with unclear follow up status; and 11.7% (interquartile range 5.6-23.7%) for participants who were explicitly not followed-up and with unclear follow-up status. When authors explicitly reported not following-up participants, they most often conducted complete case analysis (54%). Most RCTs neither reported on missing data separately for different outcomes (99%) nor reported using a method for judging risk of bias associated with missing data (95%). CONCLUSION: "Potentially missing data" are considerably more frequent than "definitely missing data." Adequate reporting of missing data will require development of explicit standards on which editors insist and to which RCT authors adhere.

Knowledge, attitudes, beliefs, values, preferences, and feasibility in relation to the use of injection safety devices in healthcare settings: a systematic review.

BACKGROUND: Adopting technologies such as injection safety devices in healthcare settings can enhance injection safety. Developing guidelines for appropriate adoption of such technologies need to consider factors beyond evidence for their health effects. The objective of this study is to systematically review the published literature for evidence among healthcare workers and patients about knowledge, attitudes, beliefs, values, preferences, and feasibility in relation to the use of injection safety devices in healthcare settings. METHODS: We included both qualitative and quantitative studies conducted with the general public, patients, and healthcare workers, administrators, or policy makers. We searched MEDLINE, EMBASE, CINHAL and CENTRAL. We used a duplicate and independent approach to title and abstract screening, full text screening, data abstraction and risk of bias assessment. RESULTS: Out of a total of 6568 identified citations, we judged fourteen studies as eligible for this systematic review. All these studies were surveys, conducted with healthcare workers in high-income countries. We did not identify any qualitative study, or a study of the general public, patients, healthcare administrators or policy makers. We did not identify any study assessing knowledge, or values assigned to outcomes relevant to injection safety devices. Each of the included studies suffered from methodological limitations, which lowers our confidence in their findings. Based on the findings of six studies, the injection safety devices were generally perceived as easy to use and as an improvement compared with conventional syringes. Some of these studies reported few technical problems while using the devices. In three studies assessing perceived safety, the majority of participants judged the devices as safe. Two studies reported positive perceptions of healthcare workers regarding patient tolerance of these injection safety devices. One study found that less than half the nurses felt comfortable using the insulin pens. Findings from four studies assessing preference and satisfaction were not consistent. CONCLUSIONS: This systematic review identified evidence that injection safety devices are generally perceived as easy to use, safe, and tolerated by patients. There were few reports of technical problems while using the devices and some discomfort by nurses using the insulin pens.

Current reperfusion strategies for ST-elevation myocardial infarction in an academic medical center in a developing country: efficacy of primary percutaneous coronary intervention.

OBJECTIVES: To study the reperfusion strategies currently being used in the treatment of ST-elevation myocardial infarction (STEMI) at an academic medical center in a developing country and to analyze the door-to-balloon time (DBT) in those patients undergoing primary percutaneous coronary intervention (PCI). METHODS: The study included all patients presenting with STEMI to the emergency department at the American University of Beirut Medical Center between July 2008 and February 2010. Data were collected prospectively from the patients' medical records. RESULTS: The study population consisted of 100 consecutive patients. Compared with an earlier study from American University of Beirut Medical Center done in 2002-2005, there was a significant increase in the utilization of primary PCI for reperfusion (81% vs. 2.5%; P < 0.001). However, the median DBT was 110 minutes, with only 30% of patients achieving a DBT ≤90 minutes. The predictors of delayed DBT (>90 minutes) were culprit lesions in the circumflex artery (P = 0.007) and delayed time from electrocardiogram to arrival in the catheterization laboratory (P < 0.001). CONCLUSIONS: There was a significant increase in the utilization of primary PCI for reperfusion of STEMI in this academic medical center in a developing country. However, achieving a target DBT ≤90 minutes was suboptimal. Future studies are needed to analyze the logistic factors associated with delayed reperfusion to institute policies and systems that can enhance the efficacy of primary PCI as a reperfusion modality in these countries.