Systematic reviews of the literature: an introduction to current methods.

Systematic reviews are a type of evidence synthesis in which authors develop explicit eligibility criteria, collect all the available studies that meet these criteria, and summarize results using reproducible methods that minimize biases and errors. Systematic reviews serve different purposes and use a different methodology than other types of evidence synthesis that include narrative reviews, scoping reviews, and overviews of reviews. Systematic reviews can address questions regarding effects of interventions or exposures, diagnostic properties of tests, and prevalence or prognosis of diseases. All rigorous systematic reviews have common processes that include: 1) determining the question and eligibility criteria, including a priori specification of subgroup hypotheses 2) searching for evidence and selecting studies, 3) abstracting data and assessing risk of bias of the included studies, 4) summarizing the data for each outcome of interest, whenever possible using meta-analyses, and 5) assessing the certainty of the evidence and drawing conclusions. There are several tools that can guide and facilitate the systematic review process, but methodological and content expertise are always necessary.

Assessing the Certainty of the Evidence in Systematic Reviews: Importance, Process, and Use.

When interpreting results and drawing conclusions, authors of systematic reviews should consider the limitations of the evidence included in their review. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach provides a framework for the explicit consideration of the limitations of the evidence included in a systematic review, and for incorporating this assessment into the conclusions. Assessments of certainty of evidence are a methodological expectation of systematic reviews. The certainty of the evidence is specific to each outcome in a systematic review, and can be rated as high, moderate, low, or very low. Because it will have an important impact, before conducting certainty of evidence, reviewers must clarify the intent of their question: are they interested in causation or association. Serious concerns regarding limitations in the study design, inconsistency, imprecision, indirectness, and publication bias can decrease the certainty of the evidence. Using an example, this article describes and illustrates the importance and the steps for assessing the certainty of evidence and drawing accurate conclusions in a systematic review.

Biomarkers to inform the management of polycystic ovary syndrome: A review of systematic reviews.

INTRODUCTION: Polycystic ovarian syndrome (PCOS) is the commonest endocrine condition affecting reproductive age women. Many biomarkers may aid assessment and management, however evidence is limited on their utility in clinical practice. We conducted a review of systematic reviews to identify the most useful biomarkers in the clinical management of PCOS. METHODS: We searched MEDLINE, EMBASE, CENTRAL and HTA until August 2023 for reviews evaluating biomarkers in PCOS women compared to healthy controls. Methodological quality was assessed using the AMSTAR2 tool. We reported pooled evidence for each biomarker with 95% confidence intervals from the most recent, up-to-date, and best quality review. RESULTS: From 3360 citations, we included 75 systematic reviews (88 biomarkers, 191,792 women). Most reviews (50/75, 67%) were moderate quality, but reported high heterogeneity (66/75, 88%). We identified 63 abnormal biomarkers in women with PCOS versus healthy controls. Of these, 22 core biomarkers could help evaluate the multisystemic impact of PCOS and inform patient management and surveillance: dehydroepiandrosterone, prolactin, sex hormone-binding globulin, total and free testosterone, anti-Mullerian hormone, systolic and diastolic blood pressure, c-reactive protein, fibrinogen, oral glucose tolerance test, homoeostatic model assessment-insulin resistance index, fasting insulin, total cholesterol, triglycerides, lipoprotein(a), HDL, LDL, non-HDL-cholesterol, ferritin, iron, and 25-hydroxy-vitamin D. CONCLUSION: We identified 22 core biomarkers assessing the multisystemic impact of PCOS and inform its clinical management. Future research is required to establish validated healthcare pathways.

Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index.

OBJECTIVE: The fragility index (FI) measures the robustness of statistically significant findings in randomized controlled trials (RCTs) by quantifying the minimum number of event conversions required to reverse a dichotomous outcome's statistical significance. In vascular surgery, many clinical guidelines and critical decision-making points are informed by a handful of key RCTs, especially regarding open surgical versus endovascular treatment. The objective of this study is to evaluate the FI of RCTs with statistically significant primary outcomes that compared open vs endovascular surgery in vascular surgery. METHODS: In this meta-epidemiological study and systematic review, MEDLINE, Embase, and CENTRAL were searched for RCTs comparing open versus endovascular treatments for abdominal aortic aneurysms, carotid artery stenosis, and peripheral arterial disease to December 2022. RCTs with statistically significant primary outcomes were included. Data screening and extraction were conducted in duplicate. The FI was calculated by adding an event to the group with the smaller number of events while subtracting a nonevent to the same group until Fisher's exact test produced a nonstatistically significant result. The primary outcome was the FI and proportion of outcomes where the loss to follow-up was greater than the FI. The secondary outcomes assessed the relationship of the FI to disease state, presence of commercial funding, and study design. RESULTS: Overall, 5133 articles were captured in the initial search with 21 RCTs reporting 23 different primary outcomes being included in the final analysis. The median FI (first quartile, third quartile) was 3 (3, 20) with 16 outcomes (70%) reporting a loss to follow-up greater than its FI. Mann-Whitney U test revealed that commercially funded RCTs and composite outcomes had greater FIs (median, 20.0 [5.5, 24.5] vs median, 3.0 [2.0, 5.5], P = .035; median, 21 [8, 38] vs median, 3.0 [2.0, 8.5], P = .01, respectively). The FI did not vary between disease states (P = .285) or between index and follow-up trials (P = .147). There were significant correlations between the FI and P values (Pearson r = 0.90; 95% confidence interval, 0.77-0.96), and the number of events (r = 0.82; 95% confidence interval, 0.48-0.97). CONCLUSIONS: A small number of event conversions (median, 3) are needed to alter the statistical significance of primary outcomes in vascular surgery RCTs evaluating open surgical and endovascular treatments. Most studies had loss to follow-up greater than its FI, which can call into question trial results, and commercially funded studies had a greater FI. The FI and these findings should be considered in future trial design in vascular surgery.

Comparing the glycaemic outcomes between real-time continuous glucose monitoring (rt-CGM) and intermittently scanned continuous glucose monitoring (isCGM) among adults and children with type 1 diabetes: A systematic review and meta-analysis of randomized controlled trials.

AIM: To conduct a systematic review and meta-analysis of randomized controlled trials (RCTs) comparing the effectiveness of real-time continuous glucose monitoring (rtCGM) versus intermittently scanned continuous glucose monitoring (isCGM) on key glycaemic metrics (co-primary outcomes HbA1c and time-in-range [TIR] 70-180 mg/dL, 3.9-10.0 mmol/L) among people with type 1 diabetes (T1D). METHODS: Medline, PubMed, Scopus, Web of Science and Cochrane Central Register of clinical trials were searched. Inclusion criteria were RCTs; T1D populations of any age and insulin regimen; comparing any type of rtCGM with isCGM (only the first generation had been compared to date); and reporting the glycaemic outcomes. Glycaemic outcomes were extracted post-intervention and expressed as mean differences and 95% CIs between the two comparators. Results were pooled using a random-effect meta-analysis. The risk of bias was assessed using the Cochrane RoB2 tool. The quality of evidence was assessed by the GRADE approach. RESULTS: Five RCTs met the inclusion criteria (4 parallel and 1 crossover design; 4 with CGM use <8 weeks), involving 446 participants (354 adults; 92 children and adolescents). Overall, meta-analysis showed rtCGM compared to isCGM improved absolute TIR by +7.0% (95% CI: 5.8%-8.3%, I2 = 0%, p < 0.01) accompanied by a favorable effect on time-below-range <70 mg/dL (3.9 mmol/L) - 1.7% (95%CI: -3.0% to -0.4%; p = 0.03). No differences were seen regarding HbA1c. CONCLUSIONS: This meta-analysis highlights that for people with T1D, rtCGM confers benefits over isCGM primarily related to increased TIR, with improvements in hypo- and hyperglycaemia.

Effectiveness of care bundles for prevention and treatment of postpartum hemorrhage: a systematic review.

OBJECTIVE: Care bundles are a promising approach to reducing postpartum hemorrhage-related morbidity and mortality. We assessed the effectiveness and safety of care bundles for postpartum hemorrhage prevention and/or treatment. DATA SOURCES: We searched MEDLINE, Embase, Cochrane CENTRAL, Maternity and Infant Care Database, and Global Index Medicus (inception to June 9, 2023) and ClinicalTrials.gov and the International Clinical Trials Registry Platform (last 5 years) using a phased search strategy, combining terms for postpartum hemorrhage and care bundles. STUDY ELIGIBILITY CRITERIA: Peer-reviewed studies evaluating postpartum hemorrhage-related care bundles were included. Care bundles were defined as interventions comprising ≥3 components implemented collectively, concurrently, or in rapid succession. Randomized and nonrandomized controlled trials, interrupted time series, and before-after studies (controlled or uncontrolled) were eligible. METHODS: Risk of bias was assessed using RoB 2 (randomized trials) and ROBINS-I (nonrandomized studies). For controlled studies, we reported risk ratios for dichotomous outcomes and mean differences for continuous outcomes, with certainty of evidence determined using GRADE. For uncontrolled studies, we used effect direction tables and summarized results narratively. RESULTS: Twenty-two studies were included for analysis. For prevention-only bundles (2 studies), low-certainty evidence suggests possible benefits in reducing blood loss, duration of hospitalization, and intensive care unit stay, and maternal well-being. For treatment-only bundles (9 studies), high-certainty evidence shows that the E-MOTIVE intervention reduced risks of composite severe morbidity (risk ratio, 0.40; 95% confidence interval, 0.32-0.50) and blood transfusion for bleeding, postpartum hemorrhage, severe postpartum hemorrhage, and mean blood loss. One nonrandomized trial and 7 uncontrolled studies suggest that other postpartum hemorrhage treatment bundles might reduce blood loss and severe postpartum hemorrhage, but this is uncertain. For combined prevention/treatment bundles (11 studies), low-certainty evidence shows that the California Maternal Quality Care Collaborative care bundle may reduce severe maternal morbidity (risk ratio, 0.64; 95% confidence interval, 0.57-0.72). Ten uncontrolled studies variably showed possible benefits, no effects, or harms for other bundle types. Nearly all uncontrolled studies did not use suitable statistical methods for single-group pretest-posttest comparisons and should thus be interpreted with caution. CONCLUSION: The E-MOTIVE intervention improves postpartum hemorrhage-related outcomes among women delivering vaginally, and the California Maternal Quality Care Collaborative bundle may reduce severe maternal morbidity. Other bundle designs warrant further effectiveness research before implementation is contemplated.

Tranexamic acid for percutaneous nephrolithotomy: an abridged Cochrane review.

OBJECTIVE: To assess the effects of tranexamic acid (TXA) in individuals with kidney stones undergoing percutaneous nephrolithotomy (PCNL). PATIENTS AND METHODS: We performed a literature search of Cochrane Library, PubMed (including MEDLINE), Embase, Scopus, Global Index Medicus, trials registries, grey literature, and conference proceedings. We included randomised controlled trials (RCTs) that compared treatment with PCNL with administration of TXA to placebo (or no TXA) for patients aged ≥18 years. Two review authors independently classified studies and abstracted data. Primary outcomes were blood transfusion, stone-free rate (SFR), thromboembolic events (TEE). We rated the certainty of evidence (CoE) according to the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach using a minimally contextualised approach with pre-defined thresholds for minimally clinically important differences (MCID). RESULTS: We included 10 RCTs assessing the effect of systemic TXA in PCNL vs placebo (or no TXA). Eight studies were published as full text. Based on an adjusted baseline risk of blood transfusion of 5.7%, systemic TXA may reduce blood transfusions (risk ratio [RR] 0.45, 95% confidence interval [CI] 0.27-0.76). Based on an adjusted baseline SFR of 75.7%, systemic TXA may increase SFR (RR 1.11, 95% CI 0.98-1.27). There is probably no difference in TEEs (risk difference 0.001, 95% CI -0.01 to 0.01). Systemic TXA may increase adverse events (AEs) (RR 5.22, 95% CI 0.52-52.72). Systemic TXA may have little to no effect on secondary interventions (RR 1.15, 95% CI 0.84-1.57). The CoE for most outcomes was assessed as low or very low. CONCLUSIONS: Based on a body of evidence of 10 RCTs, we found that systemic TXA in PCNL may reduce blood transfusions, major surgical complications, and hospital length of stay, as well as improve the SFR; however, it may increase AEs. These findings should inform urologists and their patients in making informed decisions about the use of TXA in the setting of PCNL.

Machine learning enables automated screening for systematic reviews and meta-analysis in urology.

PURPOSE: To investigate and implement semiautomated screening for meta-analyses (MA) in urology under consideration of class imbalance. METHODS: Machine learning algorithms were trained on data from three MA with detailed information of the screening process. Different methods to account for class imbalance (Sampling (up- and downsampling, weighting and cost-sensitive learning), thresholding) were implemented in different machine learning (ML) algorithms (Random Forest, Logistic Regression with Elastic Net Regularization, Support Vector Machines). Models were optimized for sensitivity. Besides metrics such as specificity, receiver operating curves, total missed studies, and work saved over sampling were calculated. RESULTS: During training, models trained after downsampling achieved the best results consistently among all algorithms. Computing time ranged between 251 and 5834 s. However, when evaluated on the final test data set, the weighting approach performed best. In addition, thresholding helped to improve results as compared to the standard of 0.5. However, due to heterogeneity of results no clear recommendation can be made for a universal sample size. Misses of relevant studies were 0 for the optimized models except for one review. CONCLUSION: It will be necessary to design a holistic methodology that implements the presented methods in a practical manner, but also takes into account other algorithms and the most sophisticated methods for text preprocessing. In addition, the different methods of a cost-sensitive learning approach can be the subject of further investigations.

Tobacco Cessation Interventions in Non-Respiratory Cancers: A Systematic Review With Meta-analysis of Randomized Controlled Trials.

BACKGROUND: Considering the high rates of persistent tobacco use, effective cessation interventions are needed for cancer patients and caregivers. Despite the need, there is a significant lack of research on tobacco cessation, especially for non-respiratory cancers (breast, prostate, colorectal, cervical, and bladder cancer). PURPOSE: The objective was to evaluate tobacco use and tobacco cessation interventions among patients and caregivers for non-respiratory cancers. METHODS: Randomized controlled trials assessing tobacco cessation interventions were identified. Five electronic databases were searched in accordance with the Preferred Reporting Items for Systematic reviews and Meta-analyses guidelines through July 2023. Studies exclusive to lung, oral, thoracic, and head and neck cancers were excluded. Effect sizes were estimated; risk of bias was assessed. RESULTS: Of 3,304 studies, 17 were included. Interventions included behavioral (n = 6), pharmacotherapy (n = 2), and a combination (n = 9) treatment. Eight studies included a health behavior model; mean behavioral change techniques were 5.57. Pooled magnitude of the odds of cessation was positive and significant (odds ratio = 1.24, 95% confidence interval [Lower Limit 1.02, Upper Limit 1.51]) relative to usual care/placebo. Cumulative meta-analysis examined the accumulation of results over-time and demonstrated that studies have been significant since 2020. Two studies included caregivers' who were involved in the provision of social support. CONCLUSIONS: Current interventions have the potential to reduce tobacco use in non-respiratory cancers. Results may be beneficial for promoting tobacco cessation among non-respiratory cancers. There is a considerable lack of dyadic interventions for cancer survivors and caregivers; researchers are encouraged to explore dyadic approaches.

We aimed to understand effective ways for cancer patients and caregivers to quit using tobacco. We focused on non-respiratory cancers (cancers not related to breathing issues) like breast, prostate, and colorectal cancer. We reviewed 17 randomized controlled trials designed to help people quit tobacco, which included behavioral therapies (e.g., education and counseling), pharmacotherapy (i.e., medicine), and combinations of both. We found that people in these studies quit using tobacco, especially when more than one approach was used. The studies also showed that these approaches have been more successful since 2020. The research highlighted a need for more studies that include both patients and their caregivers together in the quitting process. This approach, called dyadic intervention, could be more effective in supporting patients and their caregivers. Overall, while the current approaches are promising, more research is needed to develop better ways to help cancer patients and caregivers quit smoking for longer.

A protocol for an overview of systematic reviews to map photodynamic inactivation evidence in different dental specialties.

This is a protocol for an overview to summarize the findings of Systematic Reviews (SR) dealing with Photodynamic Inactivation (PDI) for control of oral diseases. Specific variables of oral infectious will be considered as outcomes, according to dental specialty. Cochrane Database of Systematic Reviews (CDSR), MEDLINE, LILACS, Embase, and Epistemonikos will be searched, as well as reference lists. A search strategy was developed for each database using only terms related to the intervention (PDI) aiming to maximize sensitivity. After checking for duplicate entries, selection of reviews will be performed in a two-stage technique: two authors will independently screening titles and abstracts, and then full texts will be assessed for inclusion/exclusion criteria. Any disagreement will be resolved through discussion and/or consultation with a third reviewer. Data will be extracted following the recommendations in Chapter V of Cochrane Handbook and using an electronic pre-specified form. The evaluation of the methodological quality and risk of bias (RoB) of the SR included will be carried out using the AMSTAR 2 and ROBIS. Narrative summaries of relevant results from the individual SR will be carried out and displayed in tables and figures. A specific summary will focus on PDI parameters and study designs, such as the type and concentration of photosensitizer, pre-irradiation time, irradiation dosimetry, and infection or microbiological models, to identify the PDI protocols with clinical potential. We will summarize the quantitative results of the SRs narratively.

Adaptive designs in clinical trials: a systematic review-part I.

BACKGROUND: Adaptive designs (ADs) are intended to make clinical trials more flexible, offering efficiency and potentially cost-saving benefits. Despite a large number of statistical methods in the literature on different adaptations to trials, the characteristics, advantages and limitations of such designs remain unfamiliar to large parts of the clinical and research community. This systematic review provides an overview of the use of ADs in published clinical trials (Part I). A follow-up (Part II) will compare the application of AD in trials in adult and pediatric studies, to provide real-world examples and recommendations for the child health community. METHODS: Published studies from 2010 to April 2020 were searched in the following databases: MEDLINE (Ovid), Embase (Ovid), and International Pharmaceutical Abstracts (Ovid). Clinical trial protocols, reports, and a secondary analyses using AD were included. We excluded trial registrations and interventions other than drugs or vaccines to align with regulatory guidance. Data from the published literature on study characteristics, types of adaptations, statistical analysis, stopping boundaries, logistical challenges, operational considerations and ethical considerations were extracted and summarized herein. RESULTS: Out of 23,886 retrieved studies, 317 publications of adaptive trials, 267 (84.2%) trial reports, and 50 (15.8%) study protocols), were included. The most frequent disease was oncology (168/317, 53%). Most trials included only adult participants (265, 83.9%),16 trials (5.4%) were limited to only children and 28 (8.9%) were for both children and adults, 8 trials did not report the ages of the included populations. Some studies reported using more than one adaptation (there were 390 reported adaptations in 317 clinical trial reports). Most trials were early in drug development (phase I, II (276/317, 87%). Dose-finding designs were used in the highest proportion of the included trials (121/317, 38.2 %). Adaptive randomization (53/317, 16.7%), with drop-the-losers (or pick-the-winner) designs specifically reported in 29 trials (9.1%) and seamless phase 2-3 design was reported in 27 trials (8.5%). Continual reassessment methods (60/317, 18.9%) and group sequential design (47/317, 14.8%) were also reported. Approximately two-thirds of trials used frequentist statistical methods (203/309, 64%), while Bayesian methods were reported in 24% (75/309) of included trials. CONCLUSION: This review provides a comprehensive report of methodological features in adaptive clinical trials reported between 2010 and 2020. Adaptation details were not uniformly reported, creating limitations in interpretation and generalizability. Nevertheless, implementation of existing reporting guidelines on ADs and the development of novel educational strategies that address the scientific, operational challenges and ethical considerations can help in the clinical trial community to decide on when and how to implement ADs in clinical trials. STUDY PROTOCOL REGISTRATION: https://doi.org/10.1186/s13063-018-2934-7 .

Features of databases that supported searching for rapid evidence synthesis during COVID-19: implications for future public health emergencies.

BACKGROUND: As evidence related to the COVID-19 pandemic surged, databases, platforms, and repositories evolved with features and functions to assist users in promptly finding the most relevant evidence. In response, research synthesis teams adopted novel searching strategies to sift through the vast amount of evidence to synthesize and disseminate the most up-to-date evidence. This paper explores the key database features that facilitated systematic searching for rapid evidence synthesis during the COVID-19 pandemic to inform knowledge management infrastructure during future global health emergencies. METHODS: This paper outlines the features and functions of previously existing and newly created evidence sources routinely searched as part of the NCCMT's Rapid Evidence Service methods, including databases, platforms, and repositories. Specific functions of each evidence source were assessed as they pertain to searching in the context of a public health emergency, including the topics of indexed citations, the level of evidence of indexed citations, and specific usability features of each evidence source. RESULTS: Thirteen evidence sources were assessed, of which four were newly created and nine were either pre-existing or adapted from previously existing resources. Evidence sources varied in topics indexed, level of evidence indexed, and specific searching functions. CONCLUSION: This paper offers insights into which features enabled systematic searching for the completion of rapid reviews to inform decision makers within 5-10 days. These findings provide guidance for knowledge management strategies and evidence infrastructures during future public health emergencies.

Designing tailored maintenance strategies for systematic reviews and clinical practice guidelines using the Portfolio Maintenance by Test-Treatment (POMBYTT) framework.

BACKGROUND: Organizations face diverse contexts and requirements when updating and maintaining their portfolio, or pool, of systematic reviews or clinical practice guidelines they need to manage. We aimed to develop a comprehensive, theoretical framework that might enable the design and tailoring of maintenance strategies for portfolios containing systematic reviews and guidelines. METHODS: We employed a conceptual approach combined with a literature review. Components of the diagnostic test-treatment pathway used in clinical healthcare were transferred to develop a framework specifically for systematic review and guideline portfolio maintenance strategies. RESULTS: We developed the Portfolio Maintenance by Test-Treatment (POMBYTT) framework comprising diagnosis, staging, management, and monitoring components. To illustrate the framework's components and their elements, we provided examples from both a clinical healthcare test-treatment pathway and a clinical practice guideline maintenance scenario. Additionally, our literature review provided possible examples for the elements in the framework, such as detection variables, detection tests, and detection thresholds. We furthermore provide three example strategies using the framework, of which one was based on living recommendations strategies. CONCLUSIONS: The developed framework might support the design of maintenance strategies that could contain multiple options besides updating to manage a portfolio (e.g. withdrawing and archiving), even in the absence of the target condition. By making different choices for variables, tests, test protocols, indications, management options, and monitoring, organizations might tailor their maintenance strategy to suit specific contexts and needs. The framework's elements could potentially aid in the design by being explicit about the operational aspects of maintenance strategies. This might also be helpful for end-users and other stakeholders of systematic reviews and clinical practice guidelines.

An analysis of published study designs in PubMed prisoner health abstracts from 1963 to 2023: a text mining study.

BACKGROUND: The challenging nature of studies with incarcerated populations and other offender groups can impede the conduct of research, particularly that involving complex study designs such as randomised control trials and clinical interventions. Providing an overview of study designs employed in this area can offer insights into this issue and how research quality may impact on health and justice outcomes. METHODS: We used a rule-based approach to extract study designs from a sample of 34,481 PubMed abstracts related to epidemiological criminology published between 1963 and 2023. The results were compared against an accepted hierarchy of scientific evidence. RESULTS: We evaluated our method in a random sample of 100 PubMed abstracts. An F1-Score of 92.2% was returned. Of 34,481 study abstracts, almost 40.0% (13,671) had an extracted study design. The most common study design was observational (37.3%; 5101) while experimental research in the form of trials (randomised, non-randomised) was present in 16.9% (2319). Mapped against the current hierarchy of scientific evidence, 13.7% (1874) of extracted study designs could not be categorised. Among the remaining studies, most were observational (17.2%; 2343) followed by systematic reviews (10.5%; 1432) with randomised controlled trials accounting for 8.7% (1196) of studies and meta-analysis for 1.4% (190) of studies. CONCLUSIONS: It is possible to extract epidemiological study designs from a large-scale PubMed sample computationally. However, the number of trials, systematic reviews, and meta-analysis is relatively small - just 1 in 5 articles. Despite an increase over time in the total number of articles, study design details in the abstracts were missing. Epidemiological criminology still lacks the experimental evidence needed to address the health needs of the marginalized and isolated population that is prisoners and offenders.

The methodological quality of systematic reviews regarding the Core Outcome Set (COS) development.

BACKGROUND: The Core Outcome Measures in Effectiveness Trials (COMET) working group proposed core outcome sets (COS) to address the heterogeneity in outcome measures in clinical studies. According to the recommendations of COMET, performing systematic reviews (SRs) usually was the first step for COS development. However, the SRs that serve as a basis for COS are not specifically appraised by organizations such as COMET regarding their quality. Here, we investigated the status of SRs related to development of COS and evaluated their methodological quality. METHODS: We conducted a search on PubMed to identify SRs related to COS development published from inception to May 2022. We qualitatively summarized the disease included in SR topics, and the studies included in the SRs. We evaluated the methodological quality of the SRs using AMSTAR 2.0 and compared the overall quality of SRs with and without protocols using the Mann-Whitney U test. RESULTS: We included 175 SRs from 23 different countries or regions, and they mainly focused on five diseases: musculoskeletal system or connective tissue disease (n = 19, 10.86%), injury, poisoning, or certain other consequences of external causes (n = 18, 10.29%), digestive system disease (n = 16, 9.14%), nervous system disease (n = 15, 8.57%), and genitourinary system disease (n = 15, 8.57%). Although 88.00% of SRs included randomized controlled trials (RCTs), only a few SRs (23.38%) employed appropriate tools to assess the risk of bias in RCTs. The assessment results on the basis of AMSTAR 2.0 indicated that most SRs (93.71%) were rated as ''critically low'' to ''low'' in terms of overall confidence. The overall confidence of SRs with protocols was significantly higher than that without protocols (P <.001). Compared to the SRs with protocols on Core Outcome Measures in Effectiveness Trials (COMET), SRs with protocols on PROSPERO were of better overall confidence (P = .017). CONCLUSION: The overall quality of published SRs regarding COS development was poor. Our findings emphasize the need for researchers to carefully select the disease topic and strictly adhere to the requirements of optimal methodology when conducting a SR for the establishment of a COS.

Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024.

PURPOSE: Although comprehensive and widespread guidelines on how to conduct systematic reviews of outcome measurement instruments (OMIs) exist, for example from the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) initiative, key information is often missing in published reports. This article describes the development of an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guideline: PRISMA-COSMIN for OMIs 2024. METHODS: The development process followed the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines and included a literature search, expert consultations, a Delphi study, a hybrid workgroup meeting, pilot testing, and an end-of-project meeting, with integrated patient/public involvement. RESULTS: From the literature and expert consultation, 49 potentially relevant reporting items were identified. Round 1 of the Delphi study was completed by 103 panelists, whereas round 2 and 3 were completed by 78 panelists. After 3 rounds, agreement (≥ 67%) on inclusion and wording was reached for 44 items. Eleven items without consensus for inclusion and/or wording were discussed at a workgroup meeting attended by 24 participants. Agreement was reached for the inclusion and wording of 10 items, and the deletion of 1 item. Pilot testing with 65 authors of OMI systematic reviews further improved the guideline through minor changes in wording and structure, finalized during the end-of-project meeting. The final checklist to facilitate the reporting of full systematic review reports contains 54 (sub)items addressing the review's title, abstract, plain language summary, open science, introduction, methods, results, and discussion. Thirteen items pertaining to the title and abstract are also included in a separate abstract checklist, guiding authors in reporting for example conference abstracts. CONCLUSION: PRISMA-COSMIN for OMIs 2024 consists of two checklists (full reports; abstracts), their corresponding explanation and elaboration documents detailing the rationale and examples for each item, and a data flow diagram. PRISMA-COSMIN for OMIs 2024 can improve the reporting of systematic reviews of OMIs, fostering their reproducibility and allowing end-users to appraise the quality of OMIs and select the most appropriate OMI for a specific application. NOTE: In order to encourage its wide dissemination this article is freely accessible on the web sites of the journals: Health and Quality of Life Outcomes; Journal of Clinical Epidemiology; Journal of Patient-Reported Outcomes; Quality of Life Research.

Systems for rating bodies of evidence used in systematic reviews of air pollution exposure and reproductive and children's health: a methodological survey.

BACKGROUND: Translating findings from systematic reviews assessing associations between environmental exposures and reproductive and children's health into policy recommendations requires valid and transparent evidence grading. METHODS: We aimed to evaluate systems for grading bodies of evidence used in systematic reviews of environmental exposures and reproductive/ children's health outcomes, by conducting a methodological survey of air pollution research, comprising a comprehensive search for and assessment of all relevant systematic reviews. To evaluate the frameworks used for rating the internal validity of primary studies and for grading bodies of evidence (multiple studies), we considered whether and how specific criteria or domains were operationalized to address reproductive/children's environmental health, e.g., whether the timing of exposure assessment was evaluated with regard to vulnerable developmental stages. RESULTS: Eighteen out of 177 (9.8%) systematic reviews used formal systems for rating the body of evidence; 15 distinct internal validity assessment tools for primary studies, and nine different grading systems for bodies of evidence were used, with multiple modifications applied to the cited approaches. The Newcastle Ottawa Scale (NOS) and the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework, neither developed specifically for this field, were the most commonly used approaches for rating individual studies and bodies of evidence, respectively. Overall, the identified approaches were highly heterogeneous in both their comprehensiveness and their applicability to reproductive/children's environmental health research. CONCLUSION: Establishing the wider use of more appropriate evidence grading methods is instrumental both for strengthening systematic review methodologies, and for the effective development and implementation of environmental public health policies, particularly for protecting pregnant persons and children.

Methodological quality of 100 recent systematic reviews of health-related outcome measurement instruments: an overview of reviews.

PURPOSE: Systematic reviews evaluating and comparing the measurement properties of outcome measurement instruments (OMIs) play an important role in OMI selection. Earlier overviews of review quality (2007, 2014) evidenced substantial concerns with regards to alignment to scientific standards. This overview aimed to investigate whether the quality of recent systematic reviews of OMIs lives up to the current scientific standards. METHODS: One hundred systematic reviews of OMIs published from June 1, 2021 onwards were randomly selected through a systematic literature search performed on March 17, 2022 in MEDLINE and EMBASE. The quality of systematic reviews was appraised by two independent reviewers. An updated data extraction form was informed by the earlier studies, and results were compared to these earlier studies' findings. RESULTS: A quarter of the reviews had an unclear research question or aim, and in 22% of the reviews the search strategy did not match the aim. Half of the reviews had an incomprehensive search strategy, because relevant search terms were not included. In 63% of the reviews (compared to 41% in 2014 and 30% in 2007) a risk of bias assessment was conducted. In 73% of the reviews (some) measurement properties were evaluated (58% in 2014 and 55% in 2007). In 60% of the reviews the data were (partly) synthesized (42% in 2014 and 7% in 2007); evaluation of measurement properties and data syntheses was not conducted separately for subscales in the majority. Certainty assessments of the quality of the total body of evidence were conducted in only 33% of reviews (not assessed in 2014 and 2007). The majority (58%) did not make any recommendations on which OMI (not) to use. CONCLUSION: Despite clear improvements in risk of bias assessments, measurement property evaluation and data synthesis, specifying the research question, conducting the search strategy and performing a certainty assessment remain poor. To ensure that systematic reviews of OMIs meet current scientific standards, more consistent conduct and reporting of systematic reviews of OMIs is needed.

Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024.

PURPOSE: Although comprehensive and widespread guidelines on how to conduct systematic reviews of outcome measurement instruments (OMIs) exist, for example from the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) initiative, key information is often missing in published reports. This article describes the development of an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guideline: PRISMA-COSMIN for OMIs 2024. METHODS: The development process followed the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines and included a literature search, expert consultations, a Delphi study, a hybrid workgroup meeting, pilot testing, and an end-of-project meeting, with integrated patient/public involvement. RESULTS: From the literature and expert consultation, 49 potentially relevant reporting items were identified. Round 1 of the Delphi study was completed by 103 panelists, whereas round 2 and 3 were completed by 78 panelists. After 3 rounds, agreement (≥ 67%) on inclusion and wording was reached for 44 items. Eleven items without consensus for inclusion and/or wording were discussed at a workgroup meeting attended by 24 participants. Agreement was reached for the inclusion and wording of 10 items, and the deletion of 1 item. Pilot testing with 65 authors of OMI systematic reviews further improved the guideline through minor changes in wording and structure, finalized during the end-of-project meeting. The final checklist to facilitate the reporting of full systematic review reports contains 54 (sub)items addressing the review's title, abstract, plain language summary, open science, introduction, methods, results, and discussion. Thirteen items pertaining to the title and abstract are also included in a separate abstract checklist, guiding authors in reporting for example conference abstracts. CONCLUSION: PRISMA-COSMIN for OMIs 2024 consists of two checklists (full reports; abstracts), their corresponding explanation and elaboration documents detailing the rationale and examples for each item, and a data flow diagram. PRISMA-COSMIN for OMIs 2024 can improve the reporting of systematic reviews of OMIs, fostering their reproducibility and allowing end-users to appraise the quality of OMIs and select the most appropriate OMI for a specific application. NOTE: In order to encourage its wide dissemination this article is freely accessible on the web sites of the journals: Health and Quality of Life Outcomes; Journal of Clinical Epidemiology; Journal of Patient-Reported Outcomes; Quality of Life Research.