Deep learning to refine the identification of high-quality clinical research articles from the biomedical literature: Performance evaluation.

BACKGROUND: Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence. OBJECTIVE: To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice. METHODS: We fine-tuned variations of BERT models (BERTBASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012 to 2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset. RESULTS: In training, three of the four selected best performing models were trained using BioBERTBASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles. CONCLUSIONS: Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.

Preliminary comparison of the performance of the National Library of Medicine's systematic review publication type and the sensitive clinical queries filter for systematic reviews in PubMed.

OBJECTIVE: The National Library of Medicine (NLM) inaugurated a "publication type" concept to facilitate searches for systematic reviews (SRs). On the other hand, clinical queries (CQs) are validated search strategies designed to retrieve scientifically sound, clinically relevant original and review articles from biomedical literature databases. We compared the retrieval performance of the SR publication type (SR[pt]) against the most sensitive CQ for systematic review articles (CQrs) in PubMed. METHODS: We ran date-limited searches of SR[pt] and CQrs to compare the relative yield of articles and SRs, focusing on the differences in retrieval of SRs by SR[pt] but not CQrs (SR[pt] NOT CQrs) and CQrs NOT SR[pt]. Random samples of articles retrieved in each of these comparisons were examined for SRs until a consistent pattern became evident. RESULTS: For SR[pt] NOT CQrs, the yield was relatively low in quantity but rich in quality, with 79% of the articles being SRs. For CQrs NOT SR[pt], the yield was high in quantity but low in quality, with only 8% being SRs. For CQrs AND SR[pt], the quality was highest, with 92% being SRs. CONCLUSIONS: We found that SR[pt] had high precision and specificity for SRs but low recall (sensitivity), whereas CQrs had much higher recall. SR[pt] OR CQrs added valid SRs to the CQrs yield at low cost (i.e., added few non-SRs). For searches that are intended to be exhaustive for SRs, SR[pt] can be added to existing sensitive search filters.

A Deep Learning Approach to Refine the Identification of High-Quality Clinical Research Articles From the Biomedical Literature: Protocol for Algorithm Development and Validation.

BACKGROUND: A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence. OBJECTIVE: The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature. METHODS: Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores. RESULTS: Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022. CONCLUSIONS: The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/29398.

Machine Learning Approaches to Retrieve High-Quality, Clinically Relevant Evidence From the Biomedical Literature: Systematic Review.

BACKGROUND: The rapid growth of the biomedical literature makes identifying strong evidence a time-consuming task. Applying machine learning to the process could be a viable solution that limits effort while maintaining accuracy. OBJECTIVE: The goal of the research was to summarize the nature and comparative performance of machine learning approaches that have been applied to retrieve high-quality evidence for clinical consideration from the biomedical literature. METHODS: We conducted a systematic review of studies that applied machine learning techniques to identify high-quality clinical articles in the biomedical literature. Multiple databases were searched to July 2020. Extracted data focused on the applied machine learning model, steps in the development of the models, and model performance. RESULTS: From 3918 retrieved studies, 10 met our inclusion criteria. All followed a supervised machine learning approach and applied, from a limited range of options, a high-quality standard for the training of their model. The results show that machine learning can achieve a sensitivity of 95% while maintaining a high precision of 86%. CONCLUSIONS: Machine learning approaches perform well in retrieving high-quality clinical studies. Performance may improve by applying more sophisticated approaches such as active learning and unsupervised machine learning approaches.

"Evidence-based checklists" for identifying predatory journals have not been assessed for reliability or validity: An analysis and proposal for moving forward.

BACKGROUND: Predatory journals (PJs) pose a threat to the quality/integrity of scientific publishing. Checklists have been proposed to identify PJs, but few are "evidence-based". This study's objective was to search for/assess evidence-based checklists (EBCs) for reliability and validity, based on a new consensus definition of PJs. METHODS: A published methods guideline for scale development was used to identify how many steps were completed in the generation of identified EBCs. Items from each EBC were compared against the consensus definition, and a list of items to be considered in the creation of a composite EBC to identify PJs was generated. RESULTS: Four EBCs were identified. None of these had completed the first of the nine steps for scale development and validation. Forty-seven items from the four EBCs were assessed against the consensus definition, of which 28 items fell within the definition. A proposed composite EBC was created from items matching components of the consensus definition. CONCLUSION: EBCs to detect PJs lack assessment of reliability and validity. To a varying degree, the EBCs contain items that match the scope of the new consensus definition of PJs. With the recent consensus definition, EBCs creators now have a clearer target, and can make adjustments.

Context and Approach in Reporting Evaluations of Electronic Health Record-Based Implementation Projects.

Electronic health records (EHRs) are ubiquitous yet still evolving, resulting in a moving target for determining the effects of context (features of the work environment, such as organization, payment systems, user training, and roles) on EHR implementation projects. Electronic health records have become instrumental in effecting quality improvement innovations and providing data to evaluate them. However, reports of studies typically fail to provide adequate descriptions of contextual details to permit readers to apply the findings. As for any evaluation, the quality of reporting is essential to learning from, and disseminating, the results. Extensive guidelines exist for reporting of virtually all types of applied health research, but they are not tailored to capture some contextual factors that may affect the outcomes of EHR implementations, such as attitudes toward implementation, format and amount of training, post go-live support, amount of local customization, and time diverted from direct interaction with patients to computers. Nevertheless, evaluators of EHR-based innovations can choose reporting guidelines that match the general purpose of their evaluation and the stage of their investigation (planning, protocol, execution, and analysis) and should report relevant contextual details (including, if pertinent, any pressures to help justify the huge investments and many years required for some implementations). Reporting guidelines are based on the scientific principles and practices that underlie sound research and should be consulted from the earliest stages of planning evaluations and onward, serving as guides for how evaluations should be conducted as well as reported.

Development and validation of study filters for identifying controlled non-randomized studies in PubMed and Ovid MEDLINE.

A retrospective analysis published by the German Institute for Quality and Efficiency in Health Care (IQWiG) in 2018 concluded that no filter for non-randomized studies (NRS) achieved sufficient sensitivity (≥92%), a precondition for comprehensive information retrieval. New NRS filters are therefore required, taking into account the challenges related to this study type. Our evaluation focused on the development of study filters for NRS with a control group ("controlled NRS"), as this study type allows the calculation of an effect size. In addition, we assumed that due to the more explicit search syntax, controlled NRS are easier to identify than non-controlled ones, potentially resulting in better performance measures of study filters for controlled NRS. Our aim was to develop study filters for identifying controlled NRS in PubMed and Ovid MEDLINE. We developed two new search filters that can assist clinicians and researchers in identifying controlled NRS in PubMed and Ovid MEDLINE. The reference set was based on 2110 publications in Medline extracted from 271 Cochrane reviews and on 4333 irrelevant references. The first filter maximizes sensitivity (92.42%; specificity 79.67%, precision 68.49%) and should be used when a comprehensive search is needed. The second filter maximizes specificity (92.06%; precision 82.98%, sensitivity 80.94%) and should be used when a more focused search is sufficient.

An evidence rating service provided valid correlates of the clinical importance of medical articles and journals.

OBJECTIVES: The objective of this study was to determine reliability and validity of McMaster PLUS measures of scientific merit and clinical importance of articles in medical journals. STUDY DESIGN AND SETTING: Analytic survey of peer-reviewed medical journals was carried out. Articles were qualified for inclusion by meeting (1) scientific criteria and (2) a clinical importance rating threshold. Included articles were sent as e-mail alerts to physicians according to their clinical interests. Internal measures included the number of high-quality, clinically important studies published in source journals and response to alerts. For external validation, we correlated internal measures with the Journal Impact Factor (JIF) and citation in DynaMed Plus (DMP). RESULTS: We evaluated 34,232 articles from 57 journals. Inclusion criteria were met by 2,638 articles (7.71%). The number of qualifying articles per journal was correlated with the number of articles with high clinical importance ratings (r 0.96, P < 0.001), article alert clicks (r 0.86, P < 0.001), and DMP citations (r 0.99, P < 0.001). Correlation was lower with the JIF (r 0.68, P < 0.01). CONCLUSIONS: Measures of scientific merit and clinical importance of medical journal articles were strongly correlated with each other, less so with JIFs. Journals varied widely by these measures but, generally, few articles were both scientifically sound and clinically important.

Effect of Patient-Centered Transitional Care Services on Clinical Outcomes in Patients Hospitalized for Heart Failure: The PACT-HF Randomized Clinical Trial.

Importance: Health care services that support the hospital-to-home transition can improve outcomes in patients with heart failure (HF). Objective: To test the effectiveness of the Patient-Centered Care Transitions in HF transitional care model in patients hospitalized for HF. Design, Setting, and Participants: Stepped-wedge cluster randomized trial of 2494 adults hospitalized for HF across 10 hospitals in Ontario, Canada, from February 2015 to March 2016, with follow-up until November 2016. Interventions: Hospitals were randomized to receive the intervention (n = 1104 patients), in which nurse-led self-care education, a structured hospital discharge summary, a family physician follow-up appointment less than 1 week after discharge, and, for high-risk patients, structured nurse homevisits and heart function clinic care were provided to patients, or usual care (n = 1390 patients), in which transitional care was left to the discretion of clinicians. Main Outcomes and Measures: Primary outcomes were hierarchically ordered as composite all-cause readmission, emergency department (ED) visit, or death at 3 months; and composite all-cause readmission or ED visit at 30 days. Secondary outcomes were B-PREPARED score for discharge preparedness (range: 0 [most prepared] to 22 [least prepared]); the 3-Item Care Transitions Measure (CTM-3) for quality of transition (range: 0 [worst transition] to 100 [best transition]); the 5-level EQ-5D version (EQ-5D-5L) for quality of life (range: 0 [dead] to 1 [full health]); and quality-adjusted life-years (QALY; range: 0 [dead] to 0.5 [full health at 6 months]). Results: Among eligible patients, all 2494 (mean age, 77.7 years; 1258 [50.4%] women) completed the trial. There was no significant difference between the intervention and usual care groups in the first primary composite outcome (545 [49.4%] vs 698 [50.2%] events, respectively; hazard ratio [HR], 0.99 [95% CI, 0.83-1.19]) or in the second primary composite outcome (304 [27.5%] vs 408 [29.3%] events, respectively; HR, 0.93 [95% CI, 0.73-1.18]). There were significant differences between the intervention and usual care groups in the secondary outcomes of mean B-PREPARED score at 6 weeks (16.6 vs 13.9; difference, 2.65 [95% CI, 1.37-3.92]; P < .001); mean CTM-3 score at 6 weeks (76.5 vs 70.3; difference, 6.16 [95% CI, 0.90-11.43]; P = .02); and mean EQ-5D-5L score at 6 weeks (0.7 vs 0.7; difference, 0.06 [95% CI, 0.01 to 0.11]; P = .02) and 6 months (0.7 vs 0.6; difference, 0.06 [95% CI, 0.01-0.12]; P = .02). There was no significant difference in mean QALY between groups at 6 months (0.3 vs 0.3; difference, 0.00 [95% CI, -0.02 to 0.02]; P = .98). Conclusions and Relevance: Among patients with HF in Ontario, Canada, implementation of a patient-centered transitional care model compared with usual care did not improve a composite of clinical outcomes. Whether this type of intervention could be effective in other health care systems or locations would require further research. Trial Registration: ClinicalTrials.gov Identifier: NCT02112227.

A Systematic Review of Medication Adherence Thresholds Dependent of Clinical Outcomes.

Background: In pharmacotherapy, the achievement of a target clinical outcome requires a certain level of medication intake or adherence. Based on Haynes's early empirical definition of sufficient adherence to antihypertensive medications as taking ≥80% of medication, many researchers used this threshold to distinguish adherent from non-adherent patients. However, we propose that different diseases, medications and patient's characteristics influence the cut-off point of the adherence rate above which the clinical outcome is satisfactory (thereafter medication adherence threshold). Moreover, the assessment of adherence and clinical outcomes may differ greatly and should be taken into consideration. To our knowledge, very few studies have defined adherence rates linked to clinical outcomes. We aimed at investigating medication adherence thresholds in relation to clinical outcomes. Method: We searched for studies that determined the relationship between adherence rates and clinical outcomes in the databases PubMed, EmbaseⓇ and Web of Science™ until December 2017, limited to English-language. Our outcome measure was any threshold value of adherence. The inclusion criteria of the retrieved studies were (1) any measurement of medication adherence, (2) any assessment of clinical outcomes, and (3) any method to define medication adherence thresholds in relation to clinical outcomes. We excluded articles considered as a tutorial. Two authors (PB and IA) independently screened titles and abstracts for relevance, reviewed full-texts, and extracted items. The results of the included studies are presented qualitatively. Result: We analyzed 6 articles that assessed clinical outcomes linked to adherence rates in 7 chronic disease states. Medication adherence was measured with Medication Possession Ratio (MPR, n = 3), Proportion of Days Covered (PDC, n = 1), both (n = 1), or Medication Event Monitoring System (MEMS). Clinical outcomes were event free episodes, hospitalization, cortisone use, reported symptoms and reduction of lipid levels. To find the relationship between the targeted clinical outcome and adherence rates, three studies applied logistic regression and three used survival analysis. Five studies defined adherence thresholds between 46 and 92%. One study confirmed the 80% threshold as valid to distinguish adherent from non-adherent patients. Conclusion: The analyzed studies were highly heterogeneous, predominantly concerning methods of calculating adherence. We could not compare studies quantitatively, mostly because adherence rates could not be standardized. Therefore, we cannot reject or confirm the validity of the historical 80% threshold. Nevertheless, the 80% threshold was clearly questioned as a general standard.

A Deep Learning Method to Automatically Identify Reports of Scientifically Rigorous Clinical Research from the Biomedical Literature: Comparative Analytic Study.

BACKGROUND: A major barrier to the practice of evidence-based medicine is efficiently finding scientifically sound studies on a given clinical topic. OBJECTIVE: To investigate a deep learning approach to retrieve scientifically sound treatment studies from the biomedical literature. METHODS: We trained a Convolutional Neural Network using a noisy dataset of 403,216 PubMed citations with title and abstract as features. The deep learning model was compared with state-of-the-art search filters, such as PubMed's Clinical Query Broad treatment filter, McMaster's textword search strategy (no Medical Subject Heading, MeSH, terms), and Clinical Query Balanced treatment filter. A previously annotated dataset (Clinical Hedges) was used as the gold standard. RESULTS: The deep learning model obtained significantly lower recall than the Clinical Queries Broad treatment filter (96.9% vs 98.4%; P<.001); and equivalent recall to McMaster's textword search (96.9% vs 97.1%; P=.57) and Clinical Queries Balanced filter (96.9% vs 97.0%; P=.63). Deep learning obtained significantly higher precision than the Clinical Queries Broad filter (34.6% vs 22.4%; P<.001) and McMaster's textword search (34.6% vs 11.8%; P<.001), but was significantly lower than the Clinical Queries Balanced filter (34.6% vs 40.9%; P<.001). CONCLUSIONS: Deep learning performed well compared to state-of-the-art search filters, especially when citations were not indexed. Unlike previous machine learning approaches, the proposed deep learning model does not require feature engineering, or time-sensitive or proprietary features, such as MeSH terms and bibliometrics. Deep learning is a promising approach to identifying reports of scientifically rigorous clinical research. Further work is needed to optimize the deep learning model and to assess generalizability to other areas, such as diagnosis, etiology, and prognosis.

Knowledge to action: Rationale and design of the Patient-Centered Care Transitions in Heart Failure (PACT-HF) stepped wedge cluster randomized trial.

INTRODUCTION: Heart Failure (HF) is a common cause of hospitalization in older adults. The transition from hospital to home is high-risk, and gaps in transitional care can increase the risk of re-hospitalization and death. Combining health care services supported by meta-analyses, we designed the PACT-HF transitional care model. METHODS: Adopting an integrated Knowledge Translation (iKT) approach in which decision-makers and clinicians are partners in research, we implement and test the effectiveness of PACT-HF among patients hospitalized for HF. We use a pragmatic stepped wedge cluster randomized trial design to introduce the complex health service intervention to 10 large hospitals in a randomized sequence until all hospitals initiate the intervention. The goal is for all patients hospitalized with HF to receive self-care education, multidisciplinary care, and early follow-up with their health care providers; and in addition, for high-risk patients to receive post-discharge nurse-led home visits and outpatient care in Heart Function clinics. This requires integration of care across hospitals, home care agencies, and outpatient clinics in our publicly funded health care system. While hospitals are the unit of recruitment and analysis, patients (estimated sample size of 3200) are the unit of analysis. Primary outcomes are hierarchically ordered as time to composite all-cause readmissions / emergency department (ED) visits / death at 3 months and time to composite all-cause readmissions / ED visits at 30 days. In a nested study of 8 hospitals, we measure the patient-centered outcomes of Discharge Preparedness, Care Transitions Quality, and Quality Adjusted Life Years (QALY); and the 6-month health care resource use and costs. We obtain all clinical and cost outcomes via linkages to provincial administrative databases. CONCLUSIONS: This protocol describes the implementation and testing of a transitional care model comprising health care services informed by high-level evidence. The study adopts an iKT and pragmatic approach, uses a robust study design, links clinical trial data with outcomes held in administrative databases, and includes patient-reported outcomes. Findings will have implications on clinical practice, health care policy, and Knowledge Translation (KT) research methodology.

Effectiveness of implementation interventions in improving physician adherence to guideline recommendations in heart failure: a systematic review.

BACKGROUND: The uptake of guideline recommendations that improve heart failure (HF) outcomes remains suboptimal. We reviewed implementation interventions that improve physician adherence to these recommendations, and identified contextual factors associated with implementation success. METHODS: We searched databases from January 1990 to November 2017 for studies testing interventions to improve uptake of class I HF guidelines. We used the Cochrane Effective Practice and Organisation of Care and Process Redesign frameworks for data extraction. Primary outcomes included: proportion of eligible patients offered guideline-recommended pharmacotherapy, self-care education, left ventricular function assessment and/or intracardiac devices. We reported clinical outcomes when available. RESULTS: We included 38 studies. Provider-level interventions (n=13 studies) included audit and feedback, reminders and education. Organisation-level interventions (n=18) included medical records system changes, multidisciplinary teams, clinical pathways and continuity of care. System-level interventions (n=3) included provider/institutional incentives. Four studies assessed multi-level interventions. We could not perform meta-analyses due to statistical/conceptual heterogeneity. Thirty-two studies reported significant improvements in at least one primary outcome. Clinical pathways, multidisciplinary teams and multifaceted interventions were most consistently successful in increasing physician uptake of guidelines. Among randomised controlled trials (RCT) (n=10), pharmacist and nurse-led interventions improved target dose prescriptions. Eleven studies reported clinical outcomes; significant improvements were reported in three, including a clinical pathway, a multidisciplinary team and a multifaceted intervention. Baseline assessment of barriers, staff training, iterative intervention development, leadership commitment and policy/financial incentives were associated with intervention effectiveness. Most studies (n=20) had medium risk of bias; nine RCTs had low risk of bias. CONCLUSION: Our study is limited by the quality and heterogeneity of the primary studies. Clinical pathways, multidisciplinary teams and multifaceted interventions appear to be most consistent in increasing guideline uptake. However, improvements in process outcomes were rarely accompanied by improvements in clinical outcomes. Our work highlights the need for improved research methodology to reliably assess the effectiveness of implementation interventions.

The efficiency of database searches for creating systematic reviews was improved by search filters.

OBJECTIVES: To compare Clinical Queries (CQs) for randomized trials of therapy 'methods' and 'NOT' limits search filters with Cochrane methods filters. STUDY DESIGN AND SETTING: Analytic survey of Cochrane reviews as the reference standard for retrieving studies included in the reviews ("included studies [ISs]"). The sensitivity and precision of Cochrane content terms + Cochrane methods terms were compared in MEDLINE and Embase with Cochrane content terms + CQs maximally sensitive filter for therapy studies, without and with additional 'NOT' limits (CQ-S [CQ sensitive]; CQ-S + limits) and a balanced filter without and with additional NOT limits (CQ-B [CQ balanced]; CQ-B + limits). RESULTS: Cochrane or CQ methods terms reduced, by 64-96%, the overall retrieval of articles with minimal loss of ISs. Sensitivity was high and similar for the 4 filters. However, CQ-B + limits had the highest precision (2.64%, number needed to be read to find one eligible study [NNR] 38) followed by the CQ-B (1.05%, NNR 95), Cochrane search (0.51%, NNR 198), CQ-S + limits (0.34%, NNR 296), and CQ-S filters (0.31%, NNR 325). CONCLUSION: For systematic reviews of therapeutic interventions, the efficiency of searches in MEDLINE and Embase was better served by the CQs for therapy studies with balanced methods filter and NOT limits.

Core Competencies in Evidence-Based Practice for Health Professionals: Consensus Statement Based on a Systematic Review and Delphi Survey.

Importance: Evidence-based practice (EBP) is necessary for improving the quality of health care as well as patient outcomes. Evidence-based practice is commonly integrated into the curricula of undergraduate, postgraduate, and continuing professional development health programs. There is, however, inconsistency in the curriculum content of EBP teaching and learning programs. A standardized set of minimum core competencies in EBP that health professionals should meet has the potential to standardize and improve education in EBP. Objective: To develop a consensus set of core competencies for health professionals in EBP. Evidence Review: For this modified Delphi survey study, a set of EBP core competencies that should be covered in EBP teaching and learning programs was developed in 4 stages: (1) generation of an initial set of relevant EBP competencies derived from a systematic review of EBP education studies for health professionals; (2) a 2-round, web-based Delphi survey of health professionals, selected using purposive sampling, to prioritize and gain consensus on the most essential EBP core competencies; (3) consensus meetings, both face-to-face and via video conference, to finalize the consensus on the most essential core competencies; and (4) feedback and endorsement from EBP experts. Findings: From an earlier systematic review of 83 EBP educational intervention studies, 86 unique EBP competencies were identified. In a Delphi survey of 234 participants representing a range of health professionals (physicians, nurses, and allied health professionals) who registered interest (88 [61.1%] women; mean [SD] age, 45.2 [10.2] years), 184 (78.6%) participated in round 1 and 144 (61.5%) in round 2. Consensus was reached on 68 EBP core competencies. The final set of EBP core competencies were grouped into the main EBP domains. For each key competency, a description of the level of detail or delivery was identified. Conclusions and Relevance: A consensus-based, contemporary set of EBP core competencies has been identified that may inform curriculum development of entry-level EBP teaching and learning programs for health professionals and benchmark standards for EBP teaching.

Development and use of a content search strategy for retrieving studies on patients' views and preferences.

BACKGROUND: Identifying scientific literature addressing patients' views and preferences is complex due to the wide range of studies that can be informative and the poor indexing of this evidence. Given the lack of guidance we developed a search strategy to retrieve this type of evidence. METHODS: We assembled an initial list of terms from several sources, including the revision of the terms and indexing of topic-related studies and, methods research literature, and other relevant projects and systematic reviews. We used the relative recall approach, evaluating the capacity of the designed search strategy for retrieving studies included in relevant systematic reviews for the topic. We implemented in practice the final version of the search strategy for conducting systematic reviews and guidelines, and calculated search's precision and the number of references needed to read (NNR). RESULTS: We assembled an initial version of the search strategy, which had a relative recall of 87.4% (yield of 132/out of 151 studies). We then added some additional terms from the studies not initially identified, and re-tested this improved version against the studies included in a new set of systematic reviews, reaching a relative recall of 85.8% (151/out of 176 studies, 95% CI 79.9 to 90.2). This final version of the strategy includes two sets of terms related with two domains: "Patient Preferences and Decision Making" and "Health State Utilities Values". When we used the search strategy for the development of systematic reviews and clinical guidelines we obtained low precision values (ranging from 2% to 5%), and the NNR from 20 to 50. CONCLUSIONS: This search strategy fills an important research gap in this field. It will help systematic reviewers, clinical guideline developers, and policy-makers to retrieve published research on patients' views and preferences. In turn, this will facilitate the inclusion of this critical aspect when formulating heath care decisions, including recommendations.

Effectiveness and content analysis of interventions to enhance medication adherence and blood pressure control in hypertension: A systematic review and meta-analysis.

OBJECTIVE: The objective of this systematic review is to evaluate the effectiveness of medication adherence interventions on blood pressure control in hypertensive patients. In addition, we aim to explore what barriers and facilitators in the interventions may have been targeted and how these might be related to the effect size on blood pressure (BP). DESIGN: This review is a hypertension-specific update to the previous Cochrane Review by Nieuwlaat et al. ( 2014 ) on interventions to enhance medication adherence. A systematic literature search was carried out and two authors independently screened titles and abstracts for their eligibility for inclusion and independently extracted data from the selected studies and assessed the methodological quality using the Cochrane Collaboration Risk of Bias Tool. A meta-analysis was conducted and additionally, theoretical factors in interventions were identified using the Theoretical Domains Framework. RESULTS: The meta-analysis found a modest main effect of adherence interventions on SBP (MD -2.71 mm Hg, 95% CI -4.17 to -1.26) and DBP (MD -1.25 mm Hg, 95% CI -1.72 to -.79). However, there was substantial significant heterogeneity across both outcomes. A narrative review on adherence outcomes was conducted. In terms of the theoretical analysis, the relationship between the total number of times the domains were coded within an intervention and change of SBP (r  =  -.234, p = .335) and DBP was not significant (r  =  -.080, p = .732). Similarly, the relationship between the total number of times different domains were coded within an intervention and change of SBP (r  =  .080, p = .746) and DBP was not significant (r  =  -.188, p = .415). DISCUSSION: This review and meta-analysis of interventions documented significant but modest post-intervention improvements in BP outcomes among hypertensive patients. However, this is a tentative finding as substantial heterogeneity and potential biases were present. One of the greatest challenges of this review was assessing risk of bias, extracting sufficient data to calculate effect size and coding interventions with the amount of information provided in papers. It is imperative that future adherence research comprehensively reports methodology.