Potential impact of missing outcome data on treatment effects in systematic reviews: imputation study.

OBJECTIVE: To assess the risk of bias associated with missing outcome data in systematic reviews. DESIGN: Imputation study. SETTING: Systematic reviews. POPULATION: 100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome. MAIN OUTCOME MEASURES: Median percentage change in the relative effect estimate when applying each of the following assumption (four commonly discussed but implausible assumptions (best case scenario, none had the event, all had the event, and worst case scenario) and four plausible assumptions for missing data based on the informative missingness odds ratio (IMOR) approach (IMOR 1.5 (least stringent), IMOR 2, IMOR 3, IMOR 5 (most stringent)); percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. Sensitivity analyses based on the eight different methods of handling missing data were conducted. RESULTS: 100 systematic reviews with 653 randomised controlled trials were included. When applying the implausible but commonly discussed assumptions, the median change in the relative effect estimate varied from 0% to 30.4%. The percentage of meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario), and 26% changed direction with the worst case scenario. When applying the plausible assumptions, the median percentage change in relative effect estimate varied from 1.4% to 7.0%. The percentage of meta-analyses crossing the threshold of the null effect varied from 6% (IMOR 1.5) to 22% (IMOR 5) of meta-analyses, and 2% changed direction with the most stringent (IMOR 5). CONCLUSION: Even when applying plausible assumptions to the outcomes of participants with definite missing data, the average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. Systematic review authors should present the potential impact of missing outcome data on their effect estimates and use this to inform their overall GRADE (grading of recommendations assessment, development, and evaluation) ratings of risk of bias and their interpretation of the results.

Meta-Analyses Proved Inconsistent in How Missing Data Were Handled Across Their Included Primary Trials: A Methodological Survey.

BACKGROUND: How systematic review authors address missing data among eligible primary studies remains uncertain. OBJECTIVE: To assess whether systematic review authors are consistent in the way they handle missing data, both across trials included in the same meta-analysis, and with their reported methods. METHODS: We first identified 100 eligible systematic reviews that included a statistically significant meta-analysis of a patient-important dichotomous efficacy outcome. Then, we successfully retrieved 638 of the 653 trials included in these systematic reviews' meta-analyses. From each trial report, we extracted statistical data used in the analysis of the outcome of interest to compare with the data used in the meta-analysis. First, we used these comparisons to classify the "analytical method actually used" for handling missing data by the systematic review authors for each included trial. Second, we assessed whether systematic reviews explicitly reported their analytical method of handling missing data. Third, we calculated the proportion of systematic reviews that were consistent in their "analytical method actually used" across trials included in the same meta-analysis. Fourth, among systematic reviews that were consistent in the "analytical method actually used" across trials and explicitly reported on a method for handling missing data, we assessed whether the "analytical method actually used" and the reported methods were consistent. RESULTS: We were unable to determine the "analytical method reviews actually used" for handling missing outcome data among 397 trials. Among the remaining 241, systematic review authors most commonly conducted "complete case analysis" (n=128, 53%) or assumed "none of the participants with missing data had the event of interest" (n=58, 24%). Only eight of 100 systematic reviews were consistent in their approach to handling missing data across included trials, but none of these reported methods for handling missing data. Among seven reviews that did explicitly report their analytical method of handling missing data, only one was consistent in their approach across included trials (using complete case analysis), and their approach was inconsistent with their reported methods (assumed all participants with missing data had the event). CONCLUSION: The majority of systematic review authors were inconsistent in their approach towards reporting and handling missing outcome data across eligible primary trials, and most did not explicitly report their methods to handle missing data. Systematic review authors should clearly identify missing outcome data among their eligible trials, specify an approach for handling missing data in their analyses, and apply their approach consistently across all primary trials.

Potentially missing data are considerably more frequent than definitely missing data: a methodological survey of 638 randomized controlled trials.

BACKGROUND AND OBJECTIVE: Missing data for the outcomes of participants in randomized controlled trials (RCTs) are a key element of risk of bias assessment. However, it is not always clear from RCT reports whether some categories of participants were followed-up or not (i.e., do or do not have missing data) nor how the RCT authors dealt with missing data in their analyses. Our objectives were to describe how RCT authors (1) report on different categories of participants that might have missing data, (2) handle these categories in the analysis, and (3) judge the risk of bias associated with missing data. METHODS: We surveyed all RCT reports included in 100 clinical intervention systematic reviews (SRs), half of which were Cochrane SRs. Eligible SRs reported a group-level meta-analysis of a patient-important dichotomous efficacy outcome, with a statistically significant effect estimate. Eleven reviewers, working in pairs, independently extracted data from the primary RCT reports included in the SRs. We predefined 19 categories of participants that might have missing data. Then, we classified these participants as follows: "explicitly followed-up," "explicitly not followed-up" (i.e., definitely missing data), or "unclear follow-up status" (i.e., potentially missing data). RESULTS: Of 638 eligible RCTs, 400 (63%) reported on at least one of the predefined categories of participants that might have missing data. The median percentage of participants who were explicitly not followed-up was 5.8% (interquartile range 2.2-14.8%); it was 9.7% (4.1-14.9%) for participants with unclear follow up status; and 11.7% (interquartile range 5.6-23.7%) for participants who were explicitly not followed-up and with unclear follow-up status. When authors explicitly reported not following-up participants, they most often conducted complete case analysis (54%). Most RCTs neither reported on missing data separately for different outcomes (99%) nor reported using a method for judging risk of bias associated with missing data (95%). CONCLUSION: "Potentially missing data" are considerably more frequent than "definitely missing data." Adequate reporting of missing data will require development of explicit standards on which editors insist and to which RCT authors adhere.

Outcomes of Cytoreductive Surgery for Metastatic Low-Grade Neuroendocrine Tumors in the Setting of Extrahepatic Metastases.

BACKGROUND: Neuroendocrine tumors (NETs) have a uniquely indolent biology. Management focuses on tumor and hormonal burden reduction. Data on cytoreduction with extrahepatic disease remain limited. OBJECTIVE: We sought to define the outcomes of cytoreduction for metastatic NETs with extrahepatic metastases. METHODS: Patients undergoing cytoreductive surgery for grade 1 or 2 NETs with extrahepatic metastases (with or without intrahepatic disease) were identified from an institutional database (2003-2014). Primary outcomes included postoperative hormonal response (> 50% urinary 5HIAA decrease), progression-free survival (PFS) and overall survival (OS), while secondary outcomes were 30-day postoperative major morbidity (Clavien grade III-V), mortality, and length of stay. RESULTS: Fifty-five patients were identified (median age 59.3 years, 80% small bowel primaries, 56.4% grade 1); 87% of patients presented with combined intra- and extrahepatic metastases. Resection most commonly included the liver (87%), small bowel (22%), mesenteric (25%) and retroperitoneal (11%) lymph nodes, and peritoneum (7%). Thirty-day major morbidity (Clavien III-V) was 18%, with 3.6% mortality, and median length of stay was 7 days [interquartile range (IQR) 5-9]. Liver embolization was performed in 31% of patients after surgery, at a median of 23 months following surgery. Overall, postoperative hormonal response occurred in 70% of patients. At median follow-up of 37 months (IQR range 22-93), 42 (76%) patients were alive and 23 (41.8%) had progressed. Five-year OS was 77% and 5-year PFS was 51%. CONCLUSION: Patients undergoing cytoreduction of metastatic well-differentiated NET in the setting of extrahepatic metastatic disease experience good tumoral control with favorable PFS and OS. Cytoreductive surgery can be safely included in the therapeutic armamentarium for NET with extrahepatic metastases.

Requirements of Clinical Journals for Authors' Disclosure of Financial and Non-Financial Conflicts of Interest: A Cross Sectional Study.

IMPORTANCE: It is unclear how medical journals address authors' financial and non-financial conflict of interest (COI). OBJECTIVE: To assess the policies of clinical journals for disclosure of financial and non-financial COI. METHODS: Cross sectional study that included both review of public documents as well as a simulation of a manuscript submission for the National Library of Medicine's "core clinical journals". The study did not involve human subjects. Investigators who abstracted the data, reviewed "instructions for authors" on the journal website and, in order to reflect the actual implementation of the COI disclosure policy, simulated the submission of a manuscript. Two individuals working in duplicate and independently to abstract information using a standardized data abstraction form, resolved disagreements by discussion or with the help of a third person. RESULTS: All but one of 117 core clinical journals had a COI policy. All journals required disclosure of financial COI pertaining to the authors and a minority (35%) asked for financial COI disclosure pertaining to the family members or authors' institution (29%). Over half required the disclosure of at least one form of non-financial COI (57%), out of which only two (3%) specifically referred to intellectual COI. Small minorities of journals (17% and 24% respectively) described a potential impact of disclosed COI and of non-disclosure of COI on the editorial process. CONCLUSION: While financial COI disclosure was well defined by the majority of the journals, many did not have clear policies on disclosure of non-financial COI, disclosure of financial COI of family members and institutions of the authors, and effect of disclosed COI or non-disclosure of COI on editorial policies.