Assessing transferability in systematic reviews of health economic evaluations - a review of methodological guidance.

OBJECTIVE: For assessing cost-effectiveness, Health Technology Assessment (HTA) organisations may use primary economic evaluations (P-HEs) or Systematic Reviews of Health Economic evaluations (SR-HEs). A prerequisite for meaningful results of SR-HEs is that the results from existing P-HEs are transferable to the decision context (e.g, HTA jurisdiction). A particularly pertinent issue is the high variability of costs and resource needs across jurisdictions. Our objective was to review the methods documents of HTA organisations and compare their recommendations on considering transferability in SR-HE. METHODS: We systematically hand searched the webpages of 158 HTA organisations for relevant methods documents from 8th January to 31st March 2019. Two independent reviewers performed searches and selected documents according to pre-defined criteria. One reviewer extracted data in standardised and piloted tables and a second reviewer checked them for accuracy. We synthesised data using tabulations and in a narrative way. RESULTS: We identified 155 potentially relevant documents from 63 HTA organisations. Of these, 7 were included in the synthesis. The included organisations have different aims when preparing a SR-HE (e.g. to determine the need for conducting their own P-HE). The recommendations vary regarding the underlying terminology (e.g. transferability/generalisability), the assessment approaches (e.g. structure), the assessment criteria and the integration in the review process. CONCLUSION: Only few HTA organisations address the assessment of transferability in their methodological recommendations for SR-HEs. Transferability considerations are related to different purposes. The assessment concepts and criteria are heterogeneous. Developing standards to consider transferability in SR-HEs is desirable.

Reporting of methods to prepare, pilot and perform data extraction in systematic reviews: analysis of a sample of 152 Cochrane and non-Cochrane reviews.

BACKGROUND: Previous research on data extraction methods in systematic reviews has focused on single aspects of the process. We aimed to provide a deeper insight into these methods by analysing a current sample of reviews. METHODS: We included systematic reviews of health interventions in humans published in English. We analysed 75 Cochrane reviews from May and June 2020 and a random sample of non-Cochrane reviews published in the same period and retrieved from Medline. We linked reviews with protocols and study registrations. We collected information on preparing, piloting, and performing data extraction and on use of software to assist review conduct (automation tools). Data were extracted by one author, with 20% extracted in duplicate. Data were analysed descriptively. RESULTS: Of the 152 included reviews, 77 reported use of a standardized extraction form (51%); 42 provided information on the type of form used (28%); 24 on piloting (16%); 58 on what data was collected (38%); 133 on the extraction method (88%); 107 on resolving disagreements (70%); 103 on methods to obtain additional data or information (68%); 52 on procedures to avoid data errors (34%); and 47 on methods to deal with multiple study reports (31%). Items were more frequently reported in Cochrane than non-Cochrane reviews. The data extraction form used was published in 10 reviews (7%). Use of software was rarely reported except for statistical analysis software and use of RevMan and GRADEpro GDT in Cochrane reviews. Covidence was the most frequent automation tool used: 18 reviews used it for study selection (12%) and 9 for data extraction (6%). CONCLUSIONS: Reporting of data extraction methods in systematic reviews is limited, especially in non-Cochrane reviews. This includes core items of data extraction such as methods used to manage disagreements. Few reviews currently use software to assist data extraction and review conduct. Our results can serve as a baseline to assess the uptake of such tools in future analyses.

Development, testing and use of data extraction forms in systematic reviews: a review of methodological guidance.

BACKGROUND: Data extraction forms link systematic reviews with primary research and provide the foundation for appraising, analysing, summarising and interpreting a body of evidence. This makes their development, pilot testing and use a crucial part of the systematic reviews process. Several studies have shown that data extraction errors are frequent in systematic reviews, especially regarding outcome data. METHODS: We reviewed guidance on the development and pilot testing of data extraction forms and the data extraction process. We reviewed four types of sources: 1) methodological handbooks of systematic review organisations (SRO); 2) textbooks on conducting systematic reviews; 3) method documents from health technology assessment (HTA) agencies and 4) journal articles. HTA documents were retrieved in February 2019 and database searches conducted in December 2019. One author extracted the recommendations and a second author checked them for accuracy. Results are presented descriptively. RESULTS: Our analysis includes recommendations from 25 documents: 4 SRO handbooks, 11 textbooks, 5 HTA method documents and 5 journal articles. Across these sources the most common recommendations on form development are to use customized or adapted standardised extraction forms (14/25); provide detailed instructions on their use (10/25); ensure clear and consistent coding and response options (9/25); plan in advance which data are needed (9/25); obtain additional data if required (8/25); and link multiple reports of the same study (8/25). The most frequent recommendations on piloting extractions forms are that forms should be piloted on a sample of studies (18/25); and that data extractors should be trained in the use of the forms (7/25). The most frequent recommendations on data extraction are that extraction should be conducted by at least two people (17/25); that independent parallel extraction should be used (11/25); and that procedures to resolve disagreements between data extractors should be in place (14/25). CONCLUSIONS: Overall, our results suggest a lack of comprehensiveness of recommendations. This may be particularly problematic for less experienced reviewers. Limitations of our method are the scoping nature of the review and that we did not analyse internal documents of health technology agencies.

Application of the PANELVIEW instrument to evaluate the guideline development process of the German polytrauma guideline.

BACKGROUND: PANELVIEW is an instrument for evaluating the appropriateness of the process, methods, and outcome of guideline development and the satisfaction of the guideline group with these steps. OBJECTIVE: To evaluate the guideline development process of the German guideline on the treatment of patients with severe/multiple injuries ('German polytrauma guideline') from the perspective of the guideline group, and to identify areas where this process may be improved in the future. METHODS: We administered PANELVIEW to the participants of the 2022 update of the German polytrauma guideline. All guideline group members, including delegates of participating medical societies, steering group members, authors of guideline chapters, the chair, and methodological lead, were invited to participate. Responses were analysed using descriptive statistics. Comments received were categorised by domains/items of the tool. RESULTS: After the first, second, and last consensus conference, the guideline group was invited via email to participate in a web-based survey. Response rates were 36% (n/N = 13/36), 40% (12/30), and 37% (20/54), respectively. The mean scores for items ranged between 5.1 and 6.9 on a scale from 1 (fully disagree) to 7 (fully agree). Items with mean scores below 6.0 were related to (1) administration, (2) consideration of patients' views, perspectives, values, and preferences, and (3) the discussion of research gaps and needs for future research. CONCLUSION: The PANELVIEW tool showed that the guideline group was satisfied with most aspects of the guideline development process. Areas for improvement of the process were identified. Strategies to improve response rates should be explored.

Safety outcomes of direct oral anticoagulants in older adults with atrial fibrillation: a systematic review and meta-analysis of (subgroup analyses from) randomized controlled trials.

Balancing stroke prevention and risk of bleeding in patients with atrial fibrillation (AF) is challenging. Direct oral anticoagulants (DOACs) are by now considered standard of care for treating patients with AF in international guidelines. Our objective was to assess the safety of long-term intake of DOACs in older adults with AF. We included RCTs in elderly (≥ 65 years) patients with AF. A systematic search in MEDLINE and EMBASE was performed on 19 April 2022. For determination of risk of bias, the RoB 2 tool was applied. We pooled outcomes using random-effects meta-analyses. The quality of evidence was assessed using GRADE. Eleven RCTs with a total of 63,374 patients were identified. Two RCTs compared apixaban with either warfarin or aspirin, four edoxaban with either placebo, aspirin, or vitamin K antagonists (VKAs), two dabigatran with warfarin and three rivaroxaban with warfarin. DOACs probably reduce mortality in elderly patients with AF (HR 0.89 95%CI 0.77 to 1.02). Low-dose DOACs likely reduce bleeding compared to VKAs (HR ranged from 0.47 to 1.01). For high-dose DOACS the risk of bleeding varied widely (HR ranged from 0.80 to 1.40). We found that low-dose DOACs probably decrease mortality in AF patients. Moreover, apixaban and probably edoxaban are associated with fewer major or clinically relevant bleeding (MCRB) events compared to VKAs. For dabigatran and rivaroxaban, the risk of MCRB varies depending on dose. Moreover, subgroup analyses indicate that in the very old (≥ 85) the risk for MCRB events might be increased when using DOACs.Registration: PROSPERO: CRD42020187876.

The effects of modifying elements of written informed consent forms for elective surgical or invasive procedures: A systematic review.

OBJECTIVE: To study the effect of modifying content and design elements within written informed-consent-forms (ICF) for patients undergoing elective surgical or invasive procedures. METHODS: We included (quasi-)randomized trials in which a modified written ICF (e.g. visual aids) was compared to a standard written ICF. We searched PubMed, Web-of-Science and PsycINFO until 08/2021. Risk of Bias was assessed. The complexity of intervention was assessed using the Intervention Complexity Assessment Tool for Systematic Reviews. RESULTS: Eleven trials with 1091 participants were eligible. Effect sizes and levels of evidence varied from trivial to moderate andthere were contradictory findings for some outcomes. Providing patients with more informationin general or specific information on risks and complications mostly increased anxiety. The use of verbal risk presentation decreased anxiety and increased satisfaction.A lower readability level decreased anxiety and improved comprehension and knowledge. CONCLUSION: Our results suggest that providing more information and addressing certain types of risks have differential effects. While more information improved knowledge, it also increased anxiety. We did not find any or only insufficient evidence for many other possible ICF modifications. PRACTICE IMPLICATIONS: When developing ICFs the differential impact of different elements on patient important outcomes should be carefully considered.

Safety of dipeptidyl peptidase-4 inhibitors in older adults with type 2 diabetes: a systematic review and meta-analysis of randomized controlled trials.

REGISTRATION: PROSPERO: CRD42020210645. INTRODUCTION: We aimed to assess the safety of dipeptidyl peptidase-4 (DPP-4) inhibitors in older patients with type 2 diabetes with inadequate glycaemic control. METHODS: We included randomized controlled trials (RCTs) in older (⩾65 years) patients with type 2 diabetes. The intervention group was randomized to treatment with any DPP-4 inhibitors. A systematic search in MEDLINE and Embase was performed in December 2020. For assessing the risk of bias, RoB 2 tool was applied. The quality of evidence was assessed using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach. We pooled outcomes using random effects meta-analyses. RESULTS: We identified 16 RCTs that included 19,317 patients with a mean age of greater than 70 years. The mean HbA1c level ranged between 7.1 and 10.0 g/dl. Adding DPP-4 inhibitors to standard care alone may increase mortality slightly [risk ratio (RR) 1.04; 95% confidence interval (CI) 0.89-1.21]. Adding DPP-4 inhibitors to standard care increases the risk for hypoglycaemia (RR 1.08; 95% CI 1.01-1.16), but difference in overall adverse events is negligible. DPP-4 inhibitors added to standard care may reduce mortality compared with sulfonylureas (RR 0.88; 95% CI 0.75-1.04). DPP-4 inhibitors probably reduce the risk for hypoglycaemia compared with sulfonylureas (magnitude of effect not quantifiable because of heterogeneity) but difference in overall adverse events is negligible. There is insufficient evidence on hospitalizations, falls, fractures, renal impairment and pancreatitis. CONCLUSION: There is no evidence that DPP-4 inhibitors in addition to standard care decrease mortality but DPP-4 inhibitors increase hypoglycaemia risk. Second-line therapy in older patients should be considered cautiously even in drugs with a good safety profile such as DPP-4 inhibitors. In case second-line treatment is necessary, DPP-4 inhibitors appear to be preferable to sulfonylureas. PLAIN LANGUAGE SUMMARY: Safety of dipeptidyl peptidase-4 inhibitors in older adults with type 2 diabetes: Introduction:: We performed the review to assess the safety of dipeptidyl peptidase-4 (DPP-4) inhibitors in older type 2 diabetes patients with blood sugar outside the normal level.Methods:: To answer the question, we searched various electronic databases. We included studies in older (⩾65 years) patients with type 2 diabetes that assessed the safety of DPP-4 inhibitors. The data from the different studies were quantitatively summarized using statistical methods. We assessed the quality of the data to judge the certainty of the findings.Results:: We identified 16 studies that included 19,317 patients with a mean age greater than 70 years. The average blood sugar level of patients in the included studies was slightly or moderately increased. Adding DPP-4 inhibitors to standard care alone may increase mortality slightly. Adding DPP-4 inhibitors to standard care increases the risk for hypoglycaemia, but difference in overall adverse events is negligible. DPP-4 inhibitors added to standard care may reduce mortality compared with sulfonylureas. DPP-4s probably reduce the risk of hypoglycaemia compared with sulfonylureas (magnitude of effect not quantifiable because of heterogeneity) but difference in overall adverse events is negligible. There is insufficient evidence on hospitalizations, falls, fractures, renal impairment and pancreatitis.Conclusion:: There is no evidence that DPP-4 inhibitors in addition to standard care decrease mortality but DPP-4 inhibitors increase the risk that blood sugar falls below normal. Adding DPP-4 inhibitorss to standard care in older patients should be considered cautiously even in drugs with a good safety profile such as DPP-4 inhibitors. In case additional treatment is necessary, DPP-4 inhibitors appear to be preferable to sulfonylureas.

Development, piloting, and evaluation of an evidence-based informed consent form for total knee arthroplasty (EvAb-Pilot): a protocol for a mixed methods study.

BACKGROUND: Practitioners frequently use informed consent forms to support the physician-patient communication and the informed consent process. Informed consent for surgery often focuses on risk centered information due to high liability risks for treatment errors. This may affect patients' anxiety of adverse events and the nocebo effect. This study focuses on the optimization of pre-surgical information on risks and complications, and at the same time reconciles these information with legal requirements. METHODS: The development, piloting, and evaluation of evidence-based informed consent forms for total knee arthroplasty (TKA) and related anesthesia procedures will follow the UK MRC Framework for developing and evaluating complex interventions. Conducting different sub-studies, we will (I) qualitatively explore the information acquisition and decision-making processes, (II) develop and pilot test evidence-based informed consent forms on the example of TKA and related anesthesia procedures, (III) conduct a monocentric interrupted time series (ITS) pilot study to evaluate the effects of evidence-based informed consent forms in comparison with standard consent forms, and (IV) perform a process evaluation to identify barriers and facilitators to the implementation of the intervention and to analyze mechanisms of impact. DISCUSSION: The evidence-based and understandable presentation of risks in informed consent forms aims at avoiding distorted risk depiction and strengthening the patients' competencies to correctly assess the risks of undergoing surgery. This might reduce negative expectations and anxiety of adverse events, which in turn might reduce the nocebo effect. At the same time, the practitioners' acceptance of evidence-based informed consent forms meeting legal requirements could be increased. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04669483 . Registered 15 December 2020. German Clinical Trials Registry, DRKS00022571 . Registered 15 December 2020.

Assessing context suitability (generalizability, external validity, applicability or transferability) of findings in evidence syntheses in healthcare-An integrative review of methodological guidance.

BACKGROUND: Evidence syntheses provide the basis for evidence-based decision making in healthcare. To judge the certainty of findings for the specific decision context evidence syntheses should consider context suitability (ie, generalizability, external validity, applicability or transferability). Our objective was to determine the status quo and to provide a comprehensive overview of existing methodological recommendations of Health Technology Assessment (HTA) and Systematic Review (SR) producing organizations in assessing context suitability of evidence on effectiveness of health care interventions. Additionally, we analyzed similarities and differences between the recommendations. METHODS: In this Integrative Review we performed a structured search for methods documents from evidence synthesis producing organizations that include recommendations on appraising context suitability in effectiveness assessments. Two reviewers independently selected documents according to predefined eligibility criteria. Data were extracted in standardized and piloted tables by one reviewer and verified by a second reviewer. We performed a thematic analysis to identify and summarize the main themes and categories regarding recommended context suitability assessments. RESULTS: We included 14 methods documents of 12 organizations in our synthesis. Assessment approaches are very heterogeneous both regarding the general concepts (eg, integration in the evidence synthesis preparation process) and the content of assessments (eg, assessment criteria). CONCLUSION: Some heterogeneity seems to be justified because of the need to tailor the assessment to different settings and medical areas. However, most differences were inexplicable. More harmonization is desirable and appears possible.