Protocol for a living evidence synthesis on variants of concern and COVID-19 vaccine effectiveness.

BACKGROUND: It is evident that COVID-19 will remain a public health concern in the coming years, largely driven by variants of concern (VOC). It is critical to continuously monitor vaccine effectiveness as new variants emerge and new vaccines and/or boosters are developed. Systematic surveillance of the scientific evidence base is necessary to inform public health action and identify key uncertainties. Evidence syntheses may also be used to populate models to fill in research gaps and help to prepare for future public health crises. This protocol outlines the rationale and methods for a living evidence synthesis of the effectiveness of COVID-19 vaccines in reducing the morbidity and mortality associated with, and transmission of, VOC of SARS-CoV-2. METHODS: Living evidence syntheses of vaccine effectiveness will be carried out over one year for (1) a range of potential outcomes in the index individual associated with VOC (pathogenesis); and (2) transmission of VOC. The literature search will be conducted up to May 2023. Observational and database-linkage primary studies will be included, as well as RCTs. Information sources include electronic databases (MEDLINE; Embase; Cochrane, L*OVE; the CNKI and Wangfang platforms), pre-print servers (medRxiv, BiorXiv), and online repositories of grey literature. Title and abstract and full-text screening will be performed by two reviewers using a liberal accelerated method. Data extraction and risk of bias assessment will be completed by one reviewer with verification of the assessment by a second reviewer. Results from included studies will be pooled via random effects meta-analysis when appropriate, or otherwise summarized narratively. DISCUSSION: Evidence generated from our living evidence synthesis will be used to inform policy making, modelling, and prioritization of future research on the effectiveness of COVID-19 vaccines against VOC.

Deep learning to refine the identification of high-quality clinical research articles from the biomedical literature: Performance evaluation.

BACKGROUND: Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence. OBJECTIVE: To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice. METHODS: We fine-tuned variations of BERT models (BERTBASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012 to 2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset. RESULTS: In training, three of the four selected best performing models were trained using BioBERTBASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles. CONCLUSIONS: Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.

Risk factors for bleeding in people living with hemophilia A and B treated with regular prophylaxis: A systematic review of the literature.

BACKGROUND: Knowledge about the risk for bleeding in patients with hemophilia (PWH) would be relevant for patients, stakeholders, and policy makers. OBJECTIVES: To perform a systematic review of the literature on risk assessment models (RAMs) and risk factors for bleeding in PWH on regular prophylaxis. METHODS: We searched Medline, Embase, the Cochrane Central Register of Controlled Trials, and the Cochrane Database of Systematic Reviews from inception through August 2019. In duplicate, reviewers screened the articles for inclusion, extracted data, and assessed the risk for bias using the Quality in Prognostic Studies (QUIPS) tool. A qualitative synthesis of the results was not performed due to high heterogeneity in risk factors, outcomes definition and measurement, and statistical analysis of the results. RESULTS: From 1843 search results, 10 studies met the inclusion criteria. No RAM for the risk for bleeding in PWH was found. Most studies included only PWH A or both PWH A and B and were conducted in North America or Europe. Only one study had a low risk for bias in all the domains. Eight categories of risk factors were identified. The risk for bleeding was increased when factor levels were lower and in people with a significant history of bleeding or who engaged in physical activities involving contact. CONCLUSIONS: Our findings suggest that plasma factor levels, history of bleeds, and physical activity should be considered for the derivation analysis when building a RAM for bleeding in PWH, and the role of other risk factors, including antithrombotic treatment and obesity, should be explored.

Accuracy and Acceptability of Wrist-Wearable Activity-Tracking Devices: Systematic Review of the Literature.

BACKGROUND: Numerous wrist-wearable devices to measure physical activity are currently available, but there is a need to unify the evidence on how they compare in terms of acceptability and accuracy. OBJECTIVE: The aim of this study is to perform a systematic review of the literature to assess the accuracy and acceptability (willingness to use the device for the task it is designed to support) of wrist-wearable activity trackers. METHODS: We searched MEDLINE, Embase, the Cochrane Central Register of Controlled Trials, and SPORTDiscus for studies measuring physical activity in the general population using wrist-wearable activity trackers. We screened articles for inclusion and, for the included studies, reported data on the studies' setting and population, outcome measured, and risk of bias. RESULTS: A total of 65 articles were included in our review. Accuracy was assessed for 14 different outcomes, which can be classified in the following categories: count of specific activities (including step counts), time spent being active, intensity of physical activity (including energy expenditure), heart rate, distance, and speed. Substantial clinical heterogeneity did not allow us to perform a meta-analysis of the results. The outcomes assessed most frequently were step counts, heart rate, and energy expenditure. For step counts, the Fitbit Charge (or the Fitbit Charge HR) had a mean absolute percentage error (MAPE) <25% across 20 studies. For heart rate, the Apple Watch had a MAPE <10% in 2 studies. For energy expenditure, the MAPE was >30% for all the brands, showing poor accuracy across devices. Acceptability was most frequently measured through data availability and wearing time. Data availability was ≥75% for the Fitbit Charge HR, Fitbit Flex 2, and Garmin Vivofit. The wearing time was 89% for both the GENEActiv and Nike FuelBand. CONCLUSIONS: The Fitbit Charge and Fitbit Charge HR were consistently shown to have a good accuracy for step counts and the Apple Watch for measuring heart rate. None of the tested devices proved to be accurate in measuring energy expenditure. Efforts should be made to reduce the heterogeneity among studies.

A Deep Learning Approach to Refine the Identification of High-Quality Clinical Research Articles From the Biomedical Literature: Protocol for Algorithm Development and Validation.

BACKGROUND: A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence. OBJECTIVE: The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature. METHODS: Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores. RESULTS: Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022. CONCLUSIONS: The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/29398.

Machine Learning Approaches to Retrieve High-Quality, Clinically Relevant Evidence From the Biomedical Literature: Systematic Review.

BACKGROUND: The rapid growth of the biomedical literature makes identifying strong evidence a time-consuming task. Applying machine learning to the process could be a viable solution that limits effort while maintaining accuracy. OBJECTIVE: The goal of the research was to summarize the nature and comparative performance of machine learning approaches that have been applied to retrieve high-quality evidence for clinical consideration from the biomedical literature. METHODS: We conducted a systematic review of studies that applied machine learning techniques to identify high-quality clinical articles in the biomedical literature. Multiple databases were searched to July 2020. Extracted data focused on the applied machine learning model, steps in the development of the models, and model performance. RESULTS: From 3918 retrieved studies, 10 met our inclusion criteria. All followed a supervised machine learning approach and applied, from a limited range of options, a high-quality standard for the training of their model. The results show that machine learning can achieve a sensitivity of 95% while maintaining a high precision of 86%. CONCLUSIONS: Machine learning approaches perform well in retrieving high-quality clinical studies. Performance may improve by applying more sophisticated approaches such as active learning and unsupervised machine learning approaches.

Optimizing a literature surveillance strategy to retrieve sound overall prognosis and risk assessment model papers.

OBJECTIVE: Our aim was to develop an efficient search strategy for prognostic studies and clinical prediction guides (CPGs), optimally balancing sensitivity and precision while independent of MeSH terms, as relying on them may miss the most current literature. MATERIALS AND METHODS: We combined 2 Hedges-based search strategies, modified to remove MeSH terms for overall prognostic studies and CPGs, and ran the search on 269 journals. We read abstracts from a random subset of retrieved references until ≥ 20 per journal were reviewed and classified them as positive when fulfilling standardized quality criteria, thereby assembling a standard dataset used to calibrate the search strategy. We determined performance characteristics of our new search strategy against the Hedges standard and performance characteristics of published search strategies against the standard dataset. RESULTS: Our search strategy retrieved 16 089 references from 269 journals during our study period. One hundred fifty-four journals yielded ≥ 20 references and ≥ 1 prognostic study or CPG. Against the Hedges standard, the new search strategy had sensitivity/specificity/precision/accuracy of 84%/80%/2%/80%, respectively. Existing published strategies tested against our standard dataset had sensitivities of 36%-94% and precision of 5%-10%. DISCUSSION: We developed a new search strategy to identify overall prognosis studies and CPGs independent of MeSH terms. These studies are important for medical decision-making, as they identify specific populations and individuals who may benefit from interventions. CONCLUSION: Our results may benefit literature surveillance and clinical guideline efforts, as our search strategy performs as well as published search strategies while capturing literature at the time of publication.

Congruence between patient characteristics and interventions may partly explain medication adherence intervention effectiveness: an analysis of 190 randomized controlled trials from a Cochrane systematic review.

OBJECTIVES: Due to the negative outcomes of medication nonadherence, interventions to improve adherence have been the focus of countless studies. The congruence between adherence-related patient characteristics and interventions may partly explain the variability of effectiveness in medication adherence studies. In their latest update of a Cochrane review reporting inconsistent effects of adherence interventions, the authors offered access to their database for subanalysis. We aimed to use this database to assess congruence between adherence-related patient characteristics and interventions and its association with intervention effects. STUDY DESIGN AND SETTING: We developed a congruence score consisting of six features related to inclusion criteria, patient characteristics at baseline, and intervention design. Two independent raters extracted and scored items from the 190 studies available in the Cochrane database. We correlated overall congruence score and individual features with intervention effects regarding adherence and clinical outcomes using Kruskal-Wallis rank sum test and Fisher's exact test. RESULTS: Interrater reliability for newly extracted data was almost perfect with a Cohen's Kappa of 0.92 [95% confidence interval (CI) = 0.89-0.94; P < 0.001]. Although present in only six studies, the inclusion of nonadherent patients was the single feature significantly associated with effective adherence interventions (P = 0.003). Moreover, effective adherence interventions were significantly associated with improved clinical outcomes (odds ratio = 6.0; 95% CI = 3.1-12.0; P < 0.0001). However, neither the overall congruence score nor any other individual feature (i.e., "determinants of nonadherence as inclusion criteria," "tailoring of interventions to the inclusion criteria," "reasons for nonadherence assessed at baseline," "adjustment of intervention to individual patient needs," and "theory-based interventions") was significantly associated with intervention effects. CONCLUSION: The presence of only six studies that included nonadherent patients and the interdependency of this feature with the remaining five might preclude a conclusive assessment of congruence between patient characteristics and adherence interventions. In order to obtain clinical benefits from effective adherence interventions, we encourage researchers to focus on the inclusion of nonadherent patients.

Development and use of a content search strategy for retrieving studies on patients' views and preferences.

BACKGROUND: Identifying scientific literature addressing patients' views and preferences is complex due to the wide range of studies that can be informative and the poor indexing of this evidence. Given the lack of guidance we developed a search strategy to retrieve this type of evidence. METHODS: We assembled an initial list of terms from several sources, including the revision of the terms and indexing of topic-related studies and, methods research literature, and other relevant projects and systematic reviews. We used the relative recall approach, evaluating the capacity of the designed search strategy for retrieving studies included in relevant systematic reviews for the topic. We implemented in practice the final version of the search strategy for conducting systematic reviews and guidelines, and calculated search's precision and the number of references needed to read (NNR). RESULTS: We assembled an initial version of the search strategy, which had a relative recall of 87.4% (yield of 132/out of 151 studies). We then added some additional terms from the studies not initially identified, and re-tested this improved version against the studies included in a new set of systematic reviews, reaching a relative recall of 85.8% (151/out of 176 studies, 95% CI 79.9 to 90.2). This final version of the strategy includes two sets of terms related with two domains: "Patient Preferences and Decision Making" and "Health State Utilities Values". When we used the search strategy for the development of systematic reviews and clinical guidelines we obtained low precision values (ranging from 2% to 5%), and the NNR from 20 to 50. CONCLUSIONS: This search strategy fills an important research gap in this field. It will help systematic reviewers, clinical guideline developers, and policy-makers to retrieve published research on patients' views and preferences. In turn, this will facilitate the inclusion of this critical aspect when formulating heath care decisions, including recommendations.

Effectiveness and content analysis of interventions to enhance medication adherence and blood pressure control in hypertension: A systematic review and meta-analysis.

OBJECTIVE: The objective of this systematic review is to evaluate the effectiveness of medication adherence interventions on blood pressure control in hypertensive patients. In addition, we aim to explore what barriers and facilitators in the interventions may have been targeted and how these might be related to the effect size on blood pressure (BP). DESIGN: This review is a hypertension-specific update to the previous Cochrane Review by Nieuwlaat et al. ( 2014 ) on interventions to enhance medication adherence. A systematic literature search was carried out and two authors independently screened titles and abstracts for their eligibility for inclusion and independently extracted data from the selected studies and assessed the methodological quality using the Cochrane Collaboration Risk of Bias Tool. A meta-analysis was conducted and additionally, theoretical factors in interventions were identified using the Theoretical Domains Framework. RESULTS: The meta-analysis found a modest main effect of adherence interventions on SBP (MD -2.71 mm Hg, 95% CI -4.17 to -1.26) and DBP (MD -1.25 mm Hg, 95% CI -1.72 to -.79). However, there was substantial significant heterogeneity across both outcomes. A narrative review on adherence outcomes was conducted. In terms of the theoretical analysis, the relationship between the total number of times the domains were coded within an intervention and change of SBP (r  =  -.234, p = .335) and DBP was not significant (r  =  -.080, p = .732). Similarly, the relationship between the total number of times different domains were coded within an intervention and change of SBP (r  =  .080, p = .746) and DBP was not significant (r  =  -.188, p = .415). DISCUSSION: This review and meta-analysis of interventions documented significant but modest post-intervention improvements in BP outcomes among hypertensive patients. However, this is a tentative finding as substantial heterogeneity and potential biases were present. One of the greatest challenges of this review was assessing risk of bias, extracting sufficient data to calculate effect size and coding interventions with the amount of information provided in papers. It is imperative that future adherence research comprehensively reports methodology.

Effectiveness and content analysis of interventions to enhance medication adherence in hypertension: a systematic review and meta-analysis protocol.

BACKGROUND: Hypertension control through pharmacological treatment has led to substantial benefits in the prevention of morbidity and mortality from cardiovascular diseases. However, evidence from a number of studies suggests that as many as 50 to 80 % of patients treated for hypertension have low adherence to their treatment regimen. The objective of this systematic review is to evaluate the effectiveness of medication adherence interventions for hypertension. In addition, we aim to explore what barriers and facilitators in the interventions may have been targeted and how these might be related to the effect size on blood pressure (BP). METHODS: This review is a hypertension-specific update to the previous Cochrane Review by Nieuwlaat et al. (2014) on interventions to enhance medication adherence. A systematic literature search will be carried out, and two authors will independently screen titles and abstracts for their eligibility for inclusion and independently extract data from the selected studies and assess the methodological quality using the Cochrane Collaboration Risk of Bias Tool. A meta-analysis will be conducted, and additionally, theoretical factors in interventions will be identified using the Theoretical Domains Framework. DISCUSSION: This review will generate new information by quantitatively evaluating the effectiveness of adherence interventions for hypertension and potentially identify which theoretical domains are associated with more effective interventions and which domains have not been the subject of intervention development. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42016033358.

Interventions for Enhancing Adherence to Antiretroviral Therapy (ART): A Systematic Review of High Quality Studies.

We sought to review the effectiveness of interventions designed to improve adherence to antiretroviral therapy (ART) from studies included in a recent Cochrane review that reported a clinical and an adherence outcome, with at least 80% follow-up for 6 months or more. Data were extracted independently and in duplicate, with an adjudicator for disagreements. Risk of bias was assessed using the Cochrane Risk of Bias tool. Of 182 relevant studies in the Cochrane review, 49 were related to ART. Statistical pooling was not warranted due to heterogeneity in interventions, participants, treatments, adherence measures and outcomes. Many studies had high risk of bias in elements of design and outcome ascertainment. Only 10 studies improved both adherence and clinical outcomes. These used the following interventions: adherence counselling (two studies); a once-daily regimen (compared to twice daily); text messaging; web-based cognitive behavioral intervention; face-to-face multi-session intensive behavioral interventions (two studies); contingency management; modified directly observed therapy; and nurse-delivered home visits combined with telephone calls. Patient-related adherence interventions were the most frequently tested. Uniform adherence measures and higher quality studies of younger populations are encouraged.

Interventions for enhancing medication adherence.

BACKGROUND: People who are prescribed self administered medications typically take only about half their prescribed doses. Efforts to assist patients with adherence to medications might improve the benefits of prescribed medications. OBJECTIVES: The primary objective of this review is to assess the effects of interventions intended to enhance patient adherence to prescribed medications for medical conditions, on both medication adherence and clinical outcomes. SEARCH METHODS: We updated searches of The Cochrane Library, including CENTRAL (via http://onlinelibrary.wiley.com/cochranelibrary/search/), MEDLINE, EMBASE, PsycINFO (all via Ovid), CINAHL (via EBSCO), and Sociological Abstracts (via ProQuest) on 11 January 2013 with no language restriction. We also reviewed bibliographies in articles on patient adherence, and contacted authors of relevant original and review articles. SELECTION CRITERIA: We included unconfounded RCTs of interventions to improve adherence with prescribed medications, measuring both medication adherence and clinical outcome, with at least 80% follow-up of each group studied and, for long-term treatments, at least six months follow-up for studies with positive findings at earlier time points. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted all data and a third author resolved disagreements. The studies differed widely according to medical condition, patient population, intervention, measures of adherence, and clinical outcomes. Pooling results according to one of these characteristics still leaves highly heterogeneous groups, and we could not justify meta-analysis. Instead, we conducted a qualitative analysis with a focus on the RCTs with the lowest risk of bias for study design and the primary clinical outcome. MAIN RESULTS: The present update included 109 new RCTs published since the previous update in January 2007, bringing the total number of RCTs to 182; we found five RCTs from the previous update to be ineligible and excluded them. Studies were heterogeneous for patients, medical problems, treatment regimens, adherence interventions, and adherence and clinical outcome measurements, and most had high risk of bias. The main changes in comparison with the previous update include that we now: 1) report a lack of convincing evidence also specifically among the studies with the lowest risk of bias; 2) do not try to classify studies according to intervention type any more, due to the large heterogeneity; 3) make our database available for collaboration on sub-analyses, in acknowledgement of the need to make collective advancement in this difficult field of research. Of all 182 RCTs, 17 had the lowest risk of bias for study design features and their primary clinical outcome, 11 from the present update and six from the previous update. The RCTs at lowest risk of bias generally involved complex interventions with multiple components, trying to overcome barriers to adherence by means of tailored ongoing support from allied health professionals such as pharmacists, who often delivered intense education, counseling (including motivational interviewing or cognitive behavioral therapy by professionals) or daily treatment support (or both), and sometimes additional support from family or peers. Only five of these RCTs reported improvements in both adherence and clinical outcomes, and no common intervention characteristics were apparent. Even the most effective interventions did not lead to large improvements in adherence or clinical outcomes. AUTHORS' CONCLUSIONS: Across the body of evidence, effects were inconsistent from study to study, and only a minority of lowest risk of bias RCTs improved both adherence and clinical outcomes. Current methods of improving medication adherence for chronic health problems are mostly complex and not very effective, so that the full benefits of treatment cannot be realized. The research in this field needs advances, including improved design of feasible long-term interventions, objective adherence measures, and sufficient study power to detect improvements in patient-important clinical outcomes. By making our comprehensive database available for sharing we hope to contribute to achieving these advances.

Adherence measurement and patient recruitment methods are poor in intervention trials to improve patient adherence.

OBJECTIVES: To develop a scale and survey the measurement of patient adherence and patient recruitment, and to explore how these methods impact the results in randomized controlled trials of interventions to improve patient adherence to medications. STUDY DESIGN: Analytic survey of a purposively selected sample of patient adherence intervention trials from a systematic review, assessing the quality of adherence measurement and patient recruitment methods. RESULTS: We identified 44 different measures of adherence, with qualities ranging from valid and objective to unreliable and subjective. The median overall quality of measures of adherence was 5 (interquartile range [IQR], 3; range, 0-9, 9 is high quality). The quality of the measures was associated with variation in the estimate of adherence (Spearman r = 0.66; 95% confidence interval: 0.39, 0.83). The median overall quality of patient recruitment methods was 2 (IQR, 1; maximum score 6, higher is better). There was no significant correlation between the power of the trial to detect an effect and the quality of the patient recruitment methods. CONCLUSION: Measurement and recruitment methods in adherence trials varied considerably, and most methods were of low quality. Adherence research could be advanced by using higher quality measures of adherence and better selection and baseline assessment of study participants.

How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks.

BACKGROUND: The consistency of treatment recommendations of evidence-based medical textbooks with more recently published evidence has not been investigated to date. Inconsistencies could affect the quality of medical care. OBJECTIVE: To determine the frequency with which topics in leading online evidence-based medical textbooks report treatment recommendations consistent with more recently published research evidence. METHODS: Summarized treatment recommendations in 200 clinical topics (ie, disease states) covered in four evidence-based textbooks--UpToDate, Physicians' Information Education Resource (PIER), DynaMed, and Best Practice--were compared with articles identified in an evidence rating service (McMaster Premium Literature Service, PLUS) since the date of the most recent topic updates in each textbook. Textbook treatment recommendations were compared with article results to determine if the articles provided different, new conclusions. From these findings, the proportion of topics which potentially require updating in each textbook was calculated. RESULTS: 478 clinical topics were assessed for inclusion to find 200 topics that were addressed by all four textbooks. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks' treatment recommendations was 23% (95% CI 17-29%) for DynaMed, 52% (95% CI 45-59%) for UpToDate, 55% (95% CI 48-61%) for PIER, and 60% (95% CI 53-66%) for Best Practice (χ(2) (3)=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131-209) for DynaMed, to 488 days (range 423-554) for PIER (P<.001 across all textbooks). CONCLUSIONS: In online evidence-based textbooks, the proportion of topics with potentially outdated treatment recommendations varies substantially.

Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes.

BACKGROUND: Computerized clinical decision support systems (CCDSSs) are claimed to improve processes and outcomes of primary preventive care (PPC), but their effects, safety, and acceptance must be confirmed. We updated our previous systematic reviews of CCDSSs and integrated a knowledge translation approach in the process. The objective was to review randomized controlled trials (RCTs) assessing the effects of CCDSSs for PPC on process of care, patient outcomes, harms, and costs. METHODS: We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews Database, Inspec, and other databases, as well as reference lists through January 2010. We contacted authors to confirm data or provide additional information. We included RCTs that assessed the effect of a CCDSS for PPC on process of care and patient outcomes compared to care provided without a CCDSS. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive. RESULTS: We added 17 new RCTs to our 2005 review for a total of 41 studies. RCT quality improved over time. CCDSSs improved process of care in 25 of 40 (63%) RCTs. Cumulative scientifically strong evidence supports the effectiveness of CCDSSs for screening and management of dyslipidaemia in primary care. There is mixed evidence for effectiveness in screening for cancer and mental health conditions, multiple preventive care activities, vaccination, and other preventive care interventions. Fourteen (34%) trials assessed patient outcomes, and four (29%) reported improvements with the CCDSS. Most trials were not powered to evaluate patient-important outcomes. CCDSS costs and adverse events were reported in only six (15%) and two (5%) trials, respectively. Information on study duration was often missing, limiting our ability to assess sustainability of CCDSS effects. CONCLUSIONS: Evidence supports the effectiveness of CCDSSs for screening and treatment of dyslipidaemia in primary care with less consistent evidence for CCDSSs used in screening for cancer and mental health-related conditions, vaccinations, and other preventive care. CCDSS effects on patient outcomes, safety, costs of care, and provider satisfaction remain poorly supported.

Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review.

BACKGROUND: Underuse and overuse of diagnostic tests have important implications for health outcomes and costs. Decision support technology purports to optimize the use of diagnostic tests in clinical practice. The objective of this review was to assess whether computerized clinical decision support systems (CCDSSs) are effective at improving ordering of tests for diagnosis, monitoring of disease, or monitoring of treatment. The outcome of interest was effect on the diagnostic test-ordering behavior of practitioners. METHODS: We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for eligible articles published up to January 2010. We included randomized controlled trials comparing the use of CCDSSs to usual practice or non-CCDSS controls in clinical care settings. Trials were eligible if at least one component of the CCDSS gave suggestions for ordering or performing a diagnostic procedure. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of test ordering outcomes. RESULTS: Thirty-five studies were identified, with significantly higher methodological quality in those published after the year 2000 (p = 0.002). Thirty-three trials reported evaluable data on diagnostic test ordering, and 55% (18/33) of CCDSSs improved testing behavior overall, including 83% (5/6) for diagnosis, 63% (5/8) for treatment monitoring, 35% (6/17) for disease monitoring, and 100% (3/3) for other purposes. Four of the systems explicitly attempted to reduce test ordering rates and all succeeded. Factors of particular interest to decision makers include costs, user satisfaction, and impact on workflow but were rarely investigated or reported. CONCLUSIONS: Some CCDSSs can modify practitioner test-ordering behavior. To better inform development and implementation efforts, studies should describe in more detail potentially important factors such as system design, user interface, local context, implementation strategy, and evaluate impact on user satisfaction and workflow, costs, and unintended consequences.

Computerized clinical decision support systems for therapeutic drug monitoring and dosing: a decision-maker-researcher partnership systematic review.

BACKGROUND: Some drugs have a narrow therapeutic range and require monitoring and dose adjustments to optimize their efficacy and safety. Computerized clinical decision support systems (CCDSSs) may improve the net benefit of these drugs. The objective of this review was to determine if CCDSSs improve processes of care or patient outcomes for therapeutic drug monitoring and dosing. METHODS: We conducted a decision-maker-researcher partnership systematic review. Studies from our previous review were included, and new studies were sought until January 2010 in MEDLINE, EMBASE, Evidence-Based Medicine Reviews, and Inspec databases. Randomized controlled trials assessing the effect of a CCDSS on process of care or patient outcomes were selected by pairs of independent reviewers. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive. RESULTS: Thirty-three randomized controlled trials were identified, assessing the effect of a CCDSS on management of vitamin K antagonists (14), insulin (6), theophylline/aminophylline (4), aminoglycosides (3), digoxin (2), lidocaine (1), or as part of a multifaceted approach (3). Cluster randomization was rarely used (18%) and CCDSSs were usually stand-alone systems (76%) primarily used by physicians (85%). Overall, 18 of 30 studies (60%) showed an improvement in the process of care and 4 of 19 (21%) an improvement in patient outcomes. All evaluable studies assessing insulin dosing for glycaemic control showed an improvement. In meta-analysis, CCDSSs for vitamin K antagonist dosing significantly improved time in therapeutic range. CONCLUSIONS: CCDSSs have potential for improving process of care for therapeutic drug monitoring and dosing, specifically insulin and vitamin K antagonist dosing. However, studies were small and generally of modest quality, and effects on patient outcomes were uncertain, with no convincing benefit in the largest studies. At present, no firm recommendation for specific systems can be given. More potent CCDSSs need to be developed and should be evaluated by independent researchers using cluster randomization and primarily assess patient outcomes related to drug efficacy and safety.

Computerized clinical decision support systems for acute care management: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes.

BACKGROUND: Acute medical care often demands timely, accurate decisions in complex situations. Computerized clinical decision support systems (CCDSSs) have many features that could help. However, as for any medical intervention, claims that CCDSSs improve care processes and patient outcomes need to be rigorously assessed. The objective of this review was to systematically review the effects of CCDSSs on process of care and patient outcomes for acute medical care. METHODS: We conducted a decision-maker-researcher partnership systematic review. MEDLINE, EMBASE, Evidence-Based Medicine Reviews databases (Cochrane Database of Systematic Reviews, DARE, ACP Journal Club, and others), and the Inspec bibliographic database were searched to January 2010, in all languages, for randomized controlled trials (RCTs) of CCDSSs in all clinical areas. We included RCTs that evaluated the effect on process of care or patient outcomes of a CCDSS used for acute medical care compared with care provided without a CCDSS. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive. RESULTS: Thirty-six studies met our inclusion criteria for acute medical care. The CCDSS improved process of care in 63% (22/35) of studies, including 64% (9/14) of medication dosing assistants, 82% (9/11) of management assistants using alerts/reminders, 38% (3/8) of management assistants using guidelines/algorithms, and 67% (2/3) of diagnostic assistants. Twenty studies evaluated patient outcomes, of which three (15%) reported improvements, all of which were medication dosing assistants. CONCLUSION: The majority of CCDSSs demonstrated improvements in process of care, but patient outcomes were less likely to be evaluated and far less likely to show positive results.