Identification of application and interpretation errors that can occur in pairwise meta-analyses in systematic reviews of interventions: a systematic review.

OBJECTIVES: To generate a bank of items describing application and interpretation errors that can arise in pairwise meta-analyses in systematic reviews of interventions. STUDY DESIGN AND SETTING: MEDLINE, Embase, and Scopus were searched to identify studies describing types of errors in meta-analyses. Descriptions of errors and supporting quotes were extracted by multiple authors. Errors were reviewed at team meetings to determine if they should be excluded, reworded, or combined with other errors, and were categorized into broad categories of errors and subcategories within. RESULTS: Fifty articles met our inclusion criteria, leading to the identification of 139 errors. We identified 25 errors covering data extraction/manipulation, 74 covering statistical analyses, and 40 covering interpretation. Many of the statistical analysis errors related to the meta-analysis model (eg, using a two-stage strategy to determine whether to select a fixed or random-effects model) and statistical heterogeneity (eg, not undertaking an assessment for statistical heterogeneity). CONCLUSION: We generated a comprehensive bank of possible errors that can arise in the application and interpretation of meta-analyses in systematic reviews of interventions. This item bank of errors provides the foundation for developing a checklist to help peer reviewers detect statistical errors.

Investigation of bias due to selective inclusion of study effect estimates in meta-analyses of nutrition research.

We aimed to explore, in a sample of systematic reviews (SRs) with meta-analyses of the association between food/diet and health-related outcomes, whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available. We randomly selected SRs of food/diet and health-related outcomes published between January 2018 and June 2019. We selected the first presented meta-analysis in each review (index meta-analysis), and extracted from study reports all study effect estimates that were eligible for inclusion in the meta-analysis. We calculated the Potential Bias Index (PBI) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 suggest selective inclusion of effect estimates more or less favourable to the intervention, respectively. We also compared the index meta-analytic estimate to the median of a randomly constructed distribution of meta-analytic estimates (i.e., the estimate expected when there is no selective inclusion). Thirty-nine SRs with 312 studies were included. The estimated PBI was 0.49 (95% CI 0.42-0.55), suggesting that the selection of study effect estimates from those reported was consistent with a process of random selection. In addition, the index meta-analytic effect estimates were similar, on average, to what we would expect to see in meta-analyses generated when there was no selective inclusion. Despite this, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta-analyses, which can help readers understand the risk of selective inclusion bias in the SRs.

Variation observed in consensus judgments between pairs of reviewers when assessing the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research.

OBJECTIVES: To evaluate the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research using the Risk Of Bias due to Missing Evidence (ROB-ME) tool and determine inter-rater agreement in assessments. STUDY DESIGN AND SETTING: We assembled a random sample of 42 meta-analyses of nutrition research. Eight assessors were randomly assigned to one of four pairs. Each pair assessed 21 randomly assigned meta-analyses, and each meta-analysis was assessed by two pairs. We calculated raw percentage agreement and chance corrected agreement using Gwet's Agreement Coefficient (AC) in consensus judgments between pairs. RESULTS: Across the eight signaling questions in the ROB-ME tool, raw percentage agreement ranged from 52% to 100%, and Gwet's AC ranged from 0.39 to 0.76. For the risk-of-bias judgment, the raw percentage agreement was 76% (95% confidence interval 60% to 92%) and Gwet's AC was 0.47 (95% confidence interval 0.14 to 0.80). In seven (17%) meta-analyses, either one or both pairs judged the risk of bias due to missing evidence as "low risk". CONCLUSION: Our findings indicated substantial variation in assessments in consensus judgments between pairs for the signaling questions and overall risk-of-bias judgments. More tutorials and training are needed to help researchers apply the ROB-ME tool more consistently.

Effect estimates can be accurately calculated with data digitally extracted from interrupted time series graphs.

Interrupted time series (ITS) studies are frequently used to examine the impact of population-level interventions or exposures. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Re-analysis of ITS may be required for inclusion in meta-analysis. While publications of ITS rarely provide raw data for re-analysis, graphs are often included, from which time series data can be digitally extracted. However, the accuracy of effect estimates calculated from data digitally extracted from ITS graphs is currently unknown. Forty-three ITS with available datasets and time series graphs were included. Time series data from each graph was extracted by four researchers using digital data extraction software. Data extraction errors were analysed. Segmented linear regression models were fitted to the extracted and provided datasets, from which estimates of immediate level and slope change (and associated statistics) were calculated and compared across the datasets. Although there were some data extraction errors of time points, primarily due to complications in the original graphs, they did not translate into important differences in estimates of interruption effects (and associated statistics). Using digital data extraction to obtain data from ITS graphs should be considered in reviews including ITS. Including these studies in meta-analyses, even with slight inaccuracy, is likely to outweigh the loss of information from non-inclusion.

Quality of the Indian clinical practice guidelines for the management of cardiovascular conditions.

To assess the quality of Indian clinical practice guidelines (CPG)s for the management of cardiovascular conditions, MEDLINE, Embase, Google Scholar and websites of relevant medical associations and government organisations were searched, from inception until August 2020, to identify Indian CPGs for the management of cardiovascular disease (CVD) conditions, produced in or between 2010 and 2019. Excluded were CPGs that were not specific to India, focused on alternative systems of medicine, of non-CVD conditions (even if they included a component of CVD), and those related to the electronic devices, cardiac biomarkers, or diagnostic procedures. Quality of the each included CPG was assessed using the AGREE II tool by four reviewers in duplicate, independently. Each AGREE II domain score and overall quality score was considered low (≤40%), moderate (40.1%-59.9%), and high (≥60%). Of the 23 CPGs included, six (26%) were reported to be adapted from other CPGs. Fourteen (61%) CPGs were produced by medical associations, six (26%) by individual authors and three (13%) by government agencies. Based on the AGREE II overall quality score, two (9%) CPGs were of high quality, four (17%) and seventeen (74%) CPGs were of moderate and low quality, respectively. Except for scope and purpose, and clarity of presentation all other domains were rated low. The quality of most Indian CPGs for managing CVD conditions assessed using the AGREE II tool was moderate-to-low. Combined efforts from different stakeholders are needed to develop, disseminate and implement high-quality CPGs while identifying and addressing barriers to their uptake to optimize patient care and improve outcomes.

Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: cross sectional meta-research study.

OBJECTIVES: To examine changes in completeness of reporting and frequency of sharing data, analytical code, and other review materials in systematic reviews over time; and factors associated with these changes. DESIGN: Cross sectional meta-research study. POPULATION: Random sample of 300 systematic reviews with meta-analysis of aggregate data on the effects of a health, social, behavioural, or educational intervention. Reviews were indexed in PubMed, Science Citation Index, Social Sciences Citation Index, Scopus, and Education Collection in November 2020. MAIN OUTCOME MEASURES: The extent of complete reporting and the frequency of sharing review materials in the systematic reviews indexed in 2020 were compared with 110 systematic reviews indexed in February 2014. Associations between completeness of reporting and various factors (eg, self-reported use of reporting guidelines, journal policies on data sharing) were examined by calculating risk ratios and 95% confidence intervals. RESULTS: Several items were reported suboptimally among 300 systematic reviews from 2020, such as a registration record for the review (n=113; 38%), a full search strategy for at least one database (n=214; 71%), methods used to assess risk of bias (n=185; 62%), methods used to prepare data for meta-analysis (n=101; 34%), and source of funding for the review (n=215; 72%). Only a few items not already reported at a high frequency in 2014 were reported more frequently in 2020. No evidence indicated that reviews using a reporting guideline were more completely reported than reviews not using a guideline. Reviews published in 2020 in journals that mandated either data sharing or inclusion of data availability statements were more likely to share their review materials (eg, data, code files) than reviews in journals without such mandates (16/87 (18%) v 4/213 (2%)). CONCLUSION: Incomplete reporting of several recommended items for systematic reviews persists, even in reviews that claim to have followed a reporting guideline. Journal policies on data sharing might encourage sharing of review materials.

Data and code availability statements in systematic reviews of interventions were often missing or inaccurate: a content analysis.

OBJECTIVES: To estimate the frequency of data and code availability statements in a random sample of systematic reviews with meta-analysis of aggregate data, summarize the content of the statements and investigate how often data and code files were shared. METHODS: We searched for systematic reviews with meta-analysis of aggregate data on the effects of a health, social, behavioral, or educational intervention that were indexed in PubMed, Education Collection via ProQuest, Scopus via Elsevier, or Social Sciences Citation Index and Science Citation Index Expanded via Web of Science during a 4-week period (between November 2, and December 2, 2020). Records were randomly sorted and screened independently by two authors until our target sample of 300 systematic reviews was reached. Two authors independently recorded whether a data or code availability statement (or both) appeared in each review and coded the content of the statements using an inductive approach. RESULTS: Of the 300 included systematic reviews with meta-analysis, 86 (29%) had a data availability statement, and seven (2%) had both a data and code availability statement. In 12/93 (13%) data availability statements, authors stated that data files were available for download from the journal website or a data repository, which we verified as being true. While 39/93 (42%) authors stated data were available upon request, 37/93 (40%) implied that sharing of data files was not necessary or applicable to them, most often because "all data appear in the article" or "no datasets were generated or analyzed". DISCUSSION: Data and code availability statements appear infrequently in systematic review manuscripts. Authors who do provide a data availability statement often incorrectly imply that data sharing is not applicable to systematic reviews. Our results suggest the need for various interventions to increase data and code sharing by systematic reviewers.

Quality of systematic reviews supporting the 2017 ACC/AHA and 2018 ESC/ESH guidelines for the management of hypertension.

OBJECTIVE: To assess the methodological and reporting quality of systematic reviews (SRs) that informed recommendations in the recent American and European hypertension guidelines. DESIGN AND SETTINGS: Meta-epidemiological study. We identified SRs that were cited for class I recommendations based on Level of Evidence-A in the 2017 American College of Cardiology/American Heart Association (ACC/AHA) and the 2018 European Society of Cardiology/European Society of Hypertension (ESC/ESH) hypertension guidelines. MAIN OUTCOME MEASURES: Methodological and reporting quality of the SRs was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) checklist and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, respectively. RESULTS: A total of 40 SRs was included in the analysis (28 from 2017 ACC/AHA; 22 from 2018 ESC/ESH and 10 were included in both). Based on the AMSTAR-2 assessment, only 7.5% SRs were found to be of high methodological quality, 47.5% were of moderate, each 22.5% were of low and critically low quality. Based on the PRISMA checklist assessment, a mean of 24 items (SD (2.76) were reported appropriately, and only five SRs reported all 27 items appropriately. CONCLUSION: Methodological and reporting quality of SRs were found to vary considerably. Lack of information on the funding source of included studies, use of a protocol, integration of risk of bias assessments while interpreting findings and reporting of excluded studies were major methodological deficiencies.

Methods used to select results to include in meta-analyses of nutrition research: A meta-research study.

OBJECTIVES: To investigate how often review authors encounter multiple results from included studies that are eligible for inclusion in a particular meta-analysis, and how often methods to select results are specified. METHODS: MEDLINE and Epistemonikos were searched (January 2018-June 2019) to identify systematic reviews with meta-analysis of the association between food/diet and health-related outcomes. A random sample of these reviews was selected, and for the first presented (index) meta-analysis, rules used to select effect estimates to include in this meta-analysis were extracted from the reviews and their protocols. All effect estimates from the primary studies that were eligible for inclusion in the index meta-analyses were extracted (e.g., when a study report presented effect estimates for blood pressure at 3 weeks and 6 weeks, both unadjusted and adjusted for covariates, and all were eligible for inclusion in a meta-analysis of the effect of red meat consumption on blood pressure, we extracted all estimates, and classified the study as having "multiplicity of results"). RESULTS: Forty-two systematic reviews with 325 studies (104 randomized, 221 non-randomized) were included; 14 reviews had a protocol. In 29% of review protocols and 69% of reviews, authors specified at least one decision rule to select effect estimates when multiple were available. In 68% of studies included in the index meta-analyses, there was at least one type of multiplicity of results. CONCLUSIONS: Authors of systematic reviews of nutrition studies should anticipate encountering multiplicity of results in the included primary studies. Specification of methods to handle multiplicity when designing reviews is therefore recommended.

Development of a checklist to detect errors in meta-analyses in systematic reviews of interventions: study protocol.

Background: Systematic reviews underpin clinical practice and policies that guide healthcare decisions. A core component of many systematic reviews is meta-analysis, which is a statistical synthesis of results across studies. Errors in the conduct and interpretation of meta-analysis can lead to incorrect conclusions regarding the benefits and harms of interventions; and studies have shown that these errors are common. Enabling peer reviewers to better detect errors in meta-analysis through the use of a checklist provides an opportunity for these errors to be rectified before publication. To our knowledge, no such checklist exists. Objective: To develop and evaluate a checklist to detect errors in pairwise meta-analyses in systematic reviews of interventions. Methods: We will undertake a four-step process to develop the checklist. First, we will undertake a systematic review of studies that have evaluated errors in the conduct and interpretation of meta-analysis to generate a bank of items to consider for the checklist. Second, we will undertake a survey of systematic review methodologists and statisticians to seek their views on which items, of the bank of items generated in step 1, are most important to include in the checklist. Third, we will hold a virtual meeting to agree upon which items to include in the checklist. Fourth, before finalising the checklist, we will pilot with editors and peer reviewers of journals. Conclusion: The developed checklist is intended to help journal editors and peer reviewers identify errors in the application and interpretation of meta-analyses in systematic reviews. Fewer errors in the conduct and improved interpretation will lead to more accurate review findings and conclusions to inform clinical practice.

The REPRISE project: protocol for an evaluation of REProducibility and Replicability In Syntheses of Evidence.

BACKGROUND: Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions. METHODS: The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4). DISCUSSION: The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.

Systematic reviews of antihypertensive drugs: A review of publication trends, characteristics, and quality.

This review presents publication trends, characteristics, and quality of systematic reviews (SRs) of randomized controlled trials (RCTs) of antihypertensive drugs (AHTDs). Between 1985 and 2017, 1,173 SRs were published, and in the last 20 years, 10, 35, and 116 were published in the year 1996, 2006, and 2016, respectively. Angiotensin-converting enzyme inhibitors/angiotensin II receptor blockers were the most common class of drugs studied. Fourteen percent of the SRs were prospectively registered/published protocol. Three-fourth of the SRs did not report a full search strategy, and 45% did not report a PRISMA or similar diagram. Of the 34 SRs published in the five high impact factor journals in the last 10 years, 15%, 21%, and 65% have unclear, low, and high risk of bias, respectively. There has been a steady increase in the publication of SRs of RCTs of AHTDs. However, adherence to standard methods of conduct and reporting continues to be low.

Pharmacokinetics of Rosuvastatin: A Systematic Review of Randomised Controlled Trials in Healthy Adults.

BACKGROUND: Rosuvastatin is a lipid-lowering drug that works by inhibiting 3-hydroxy-3-methylglutaryl coenzyme A reductase, the rate-limiting enzyme responsible for producing cholesterol in humans. The pharmacokinetic data of rosuvastatin are considerably variable across studies. OBJECTIVE: To review the pharmacokinetics of rosuvastatin from randomised controlled trials (RCTs) in healthy adults. METHODS: A review of the pharmacokinetics of rosuvastatin was performed using systematic search strategies. The Sheiner method was used to summarise the pharmacokinetics of the drug. RESULTS: Randomised controlled studies (n = 70) involving healthy subjects (n = 2355) that examined the pharmacokinetics of rosuvastatin following single and multiple doses were included in the review. Rosuvastatin is given once daily in the dose range of 5-80 mg, with 40 mg being the maximum approved daily dose. Rosuvastatin achieves maximum plasma concentration at a median of 5 h (range: 0.5-6 h) under fasting conditions following single and multiple doses. Following single doses, rosuvastatin has a mean absolute oral availability of 20%, an overall mean total clearance of 28.3 L/h and an average terminal elimination half-life of approximately 20 h. The overall mean total clearance of the drug in Caucasian subjects was 1.7-fold higher than that in healthy Chinese subjects. The systemic exposure of rosuvastatin is characterised by a large coefficient of variation (48%.) There is a small accumulation with repeated dosing. The interaction of rosuvastatin with darunavir/ritonavir was considered statistically and clinically relevant. Interactions of rosuvastatin single doses with erythromycin, fluconazole, itraconazole and antacid were statistically significant. DISCUSSION AND CONCLUSIONS: There is considerable variation in the pharmacokinetics of rosuvastatin between races. The clinical relevance of the statistically significant drug interactions is yet to be investigated following repeated co-administration for at least 15 days, consistent with a half-life of low-density lipoprotein of 3 days.

Two-drug fixed-dose combinations of blood pressure-lowering drugs as WHO essential medicines: An overview of efficacy, safety, and cost.

Cardiovascular diseases (CVD) are the world's leading cause of death. High blood pressure (BP) is the leading global risk factor for all-cause preventable morbidity and mortality. Globally, only about 14% of patients achieve BP control to systolic BP <140 mm Hg and diastolic BP <90 mm Hg. Most patients (>60%) require two or more drugs to achieve BP control, yet poor adherence to therapy is a major barrier to achieving this control. Fixed-dose combinations (FDCs) of BP-lowering drugs are one means to improve BP control through greater adherence and efficacy, with favorable safety and cost profiles. The authors present a review of the supporting data from a successful application to the World Health Organization (WHO) for the inclusion of FDCs of two BP-lowering drugs on the 21st WHO Essential Medicines List. The authors discuss the efficacy and safety of FDCs of two BP-lowering drugs for the management of hypertension in adults, relevant hypertension guideline recommendations, and the estimated cost of such therapies.

Development, validation and evaluation of a novel self-instructional module in patients with chronic non-specific low back pain.

BACKGROUND: Low back pain (LBP) is ranked highest in terms of disability-adjusted life-years lived. Patient education and self-management have shown to play a crucial role in the overall pain management. However, the literature on the same with respect to Indian context is still lacking. The study was aimed to develop, validate and assess the acceptability and effectiveness of self-instructional educational module among Indian chronic LBP (CLBP) patients. METHODS: A prospective single-arm open-label study was conducted in a pain clinic of a tertiary care public hospital in North India with 'Backcare booklet-self-instructional module (SIM)' as an intervention in patients with CLBP. SIM was developed with the intent to provide up-to-date evidence-based information in an easy understanding way to patients with CLBP. 132 patients were administered SIM with a single session of verbal explanation. Pain intensity (numeric rating scale [NRS]), disability, fear-avoidance belief Questionnaire (FABQ), quality of life (EQ5D) and knowledge level were assessed at baseline and after 3 months of intervention. Student's paired t-test and Chi-square test were used. Data were analysed using SPSS version 15.0. RESULTS: 120 patients successfully completed the 3 months' follow-up. Significant reductions were observed in pain intensity (76[12] vs 55 [15, P < 0.01); disability (51[14] vs 43 [10], P < 0.01); FABQ (46[12] vs 41 [10], P < 0.01); EQ5D (0.35 [0.27] vs 0.18 [0.26], P < 0.01). CONCLUSION: Backcare booklet as an intervention, along with usual pharmacological care is a cost-effective educational medium to promote self-management of CLBP in the clinical outpatient settings.

Does Co-administration of Antihypertensive Drugs and Statins Alter Their Efficacy and Safety? A Systematic Review and Meta-analysis of Randomized Controlled Trials.

Antihypertensive drugs (AHTDs) and statins are frequently administered together, but there is uncertainty on whether the presence of one affects the main effects of the other. This systematic review and meta-analysis assessed the effects of co-administered AHTDs and statins on blood pressure (BP) and cholesterol. MEDLINE, Cochrane Central Register of Controlled Trials and drug regulatory agency websites were searched, until January 2018. Twelve double-blind randomized controlled trials that allocated adults with or without hypertension and/or hyperlipidemia (n = 4434) to fixed doses of AHTD alone, statin alone and both drugs together, for ≥4 weeks, were included. BP lowering was similar with AHTD + statin compared with AHTD alone [systolic BP -0.1 mm Hg, 95% confidence interval (CI), -1.0 to 0.8, and diastolic BP -1.0 mm Hg, 95% CI, -2.3 to -0.2]. AHTD + statin compared with statin alone resulted in small reduction in low-density lipoprotein cholesterol (-3.9 mg/dL, 95% CI, -6.1 to -1.7), and this effect was largely associated with co-administration of amlodipine and atorvastatin or rosuvastatin. There was no difference in safety outcomes. Overall, it can be concluded that there is no clinically important difference in the effects of AHTDs and statins whether used separately or together for reduction in BP and low-density lipoprotein cholesterol.

Efficacy and safety of dual combination therapy of blood pressure-lowering drugs as initial treatment for hypertension: a systematic review and meta-analysis of randomized controlled trials.

OBJECTIVE: To assess the efficacy and tolerability of dual combination of blood pressure (BP)-lowering drugs as initial treatment for hypertension. METHODS: MEDLINE, Embase, CENTRAL were searched until August 2017 for randomized, double-blind trials of dual combination therapy vs. monotherapy in adults with hypertension who were either treatment naïve or untreated for at least 4 weeks. Regimens were classified with reference to usual daily 'standard-dose'; for example, <1 + <1 for a combination of two drugs both at less than one standard-dose. Random-effects models were used for meta-analysis. RESULTS: Thirty-three trials (13 095 participants) with mean baseline mean BP 155/100 mmHg were included. Compared with standard-dose monotherapy, dual combinations of <1 + <1, 1 + <1 and 1 + 1 (i.e. low-to-standard dose), showed a dose-response relationship in reducing SBP [mean differences (95% confidence interval) of 2.8 (1.6-4.0), 4.6 (3.4-5.7) and 7.5 (5.4-9.5) mmHg, respectively], and in improving BP control [risk ratio (RR) (95% confidence interval) 1.11 (0.92-1.34), 1.25 (1.16-1.35) and 1.42 (1.27-1.58), respectively]. Withdrawals due to adverse events were uncommon with low-to-standard dose dual combinations, with no significant difference compared with standard-dose monotherapy [2.9 vs. 2.2%; RR 1.28 (0.85 to 1.92)]. There were fewer data for higher dose dual combinations, which did not appear to produce substantial additional efficacy and could potentially be less tolerable. CONCLUSION: Compared with standard-dose monotherapy, initiating treatment with low-to-standard dose dual combination therapy is more efficacious without increasing withdrawals due to adverse events. PROSPERO REGISTRATION: CRD42016032822.