A Tool to Assess the Trustworthiness of Evidence-Based Point-of-Care Information for Health Care Professionals (CAPOCI): Design and Validation Study.

BACKGROUND: User-friendly information at the point of care for health care professionals should be well structured, rapidly accessible, comprehensive, and trustworthy. The reliability of information and the associated methodological process must be clear. There is no standard tool to evaluate the trustworthiness of such point-of-care (POC) information. OBJECTIVE: We aim to develop and validate a new tool for assessment of trustworthiness of evidence-based POC resources to enhance the quality of POC resources and facilitate evidence-based practice. METHODS: We designed the Critical Appraisal of Point-of-Care Information (CAPOCI) tool based on the criteria important for assessment of trustworthiness of POC information, reported in a previously published review. A group of health care professionals and methodologists (the authors of this paper) defined criteria for the CAPOCI tool in an iterative process of discussion and pilot testing until consensus was reached. In the next step, all criteria were subject to content validation with a Delphi study. We invited an international panel of 10 experts to rate their agreement with the relevance and wording of the criteria and to give feedback. Consensus was reached when 70% of the experts agreed. When no consensus was reached, we reformulated the criteria based on the experts' comments for a next round of the Delphi study. This process was repeated until consensus was reached for each criterion. In a last step, the interrater reliability of the CAPOCI tool was calculated with a 2-tailed Kendall tau correlation coefficient to quantify the agreement between 2 users who piloted the CAPOCI tool on 5 POC resources. Two scoring systems were tested: a 3-point ordinal scale and a 7-point Likert scale. RESULTS: After validation, the CAPOCI tool was designed with 11 criteria that focused on methodological quality and author-related information. The criteria assess authorship, literature search, use of preappraised evidence, critical appraisal of evidence, expert opinions, peer review, timeliness and updating, conflict of interest, and commercial support. Interrater agreement showed substantial agreement between 2 users for scoring with the 3-point ordinal scale (τ=.621, P<.01) and scoring with the 7-point Likert scale (τ=.677, P<.01). CONCLUSIONS: The CAPOCI tool may support validation teams in the assessment of trustworthiness of POC resources. It may also provide guidance for producers of POC resources.

Tools to Assess the Trustworthiness of Evidence-Based Point-of-Care Information for Health Care Professionals: Systematic Review.

BACKGROUND: User-friendly information at the point of care should be well structured, rapidly accessible, and comprehensive. Also, this information should be trustworthy, as it will be used by health care practitioners to practice evidence-based medicine. Therefore, a standard, validated tool to evaluate the trustworthiness of such point-of-care information resources is needed. OBJECTIVE: This systematic review sought to search for tools to assess the trustworthiness of point-of-care resources and to describe and analyze the content of these tools. METHODS: A systematic search was performed on three sources: (1) we searched online for initiatives that worked off of the trustworthiness of medical information; (2) we searched Medline (PubMed) until June 2019 for relevant literature; and (3) we scanned reference lists and lists of citing papers via Web of Science for each retrieved paper. We included all studies, reports, websites, or methodologies that reported on tools that assessed the trustworthiness of medical information for professionals. From the selected studies, we extracted information on the general characteristics of the tools. As no standard, risk-of-bias assessment instruments are available for these types of studies, we described how each tool was developed, including any assessments on reliability and validity. We analyzed the criteria used in the different tools and divided them into five categories: (1) author-related information; (2) evidence-based methodology; (3) website quality; (4) website design and usability; and (5) website interactivity. The percentage of tools in compliance with these categories and the different criteria were calculated. RESULTS: Included in this review was a total of 17 tools, all published between 1997 and 2018. The tools were developed for different purposes, from a general quality assessment of medical information to very detailed analyses, all specifically for point-of-care resources. However, the development process of the tools was poorly described. Overall, seven tools had a scoring system implemented, two were assessed for reliability only, and two other tools were assessed for both validity and reliability. The content analysis showed that all the tools assessed criteria related to an evidence-based methodology: 82% of the tools assessed author-related information, 71% assessed criteria related to website quality, 71% assessed criteria related to website design and usability, and 47% of the tools assessed criteria related to website interactivity. There was significant variability in criteria used, as some were very detailed while others were more broadly defined. CONCLUSIONS: The 17 included tools encompass a variety of items important for the assessment of the trustworthiness of point-of-care information. Overall, two tools were assessed for both reliability and validity, but they lacked some essential criteria for the assessment of the trustworthiness of medical information for use at the point-of-care. Currently, a standard, validated tool does not exist. The results of this review may contribute to the development of such an instrument, which may enhance the quality of point-of-care information in the long term. TRIAL REGISTRATION: PROSPERO CRD42019122565; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=122565.

Involving general practice trainees in clinical practice guideline adaptation.

BACKGROUND: It is unclear whether it is feasible to involve residents in guideline development or adaptation. We designed a multifaceted training program that combines training sessions, a handbook and a documentation tool to assist general practice (GP)-trainees in the adaptation of clinical practice guidelines (CPGs). The aim of this study is to adapt a database of CPGs by involving GP-trainees and to build evidence-based practice (EBP) learning capacity. METHODS: We assessed each adaptation process and surveyed all GP-trainees who enrolled in our training program on their views on the program. They were asked to formulate an overall rating for the training and were asked to rate individual aspects of the training program (the training sessions, the handbook and the documentation tool). RESULTS: To date, 122 GP-trainees followed the training and have adapted 60 different CPGs. Overall quality of their work was good. Based on an assessment of the content of the documentation tool, 24 (40%) adapted CPGs rated as good quality and 30 (50%) rated as moderate quality. Only 3 adapted CPGs (5%) were evaluated as being of poor quality. 51 (42%) GP-trainees completed the survey on user satisfaction. 98% (50) of the GP-trainees found the training to be of good overall quality. 86% of the GP-trainees were satisfied with the handbook but satisfaction was lowest for the documentation tool (47% satisfied). CONCLUSION: It is possible to engage GP-trainees in CPG adaptation using a formal process when provided with training, feedback and documentation tools.

Continuing medical education for general practitioners: a practice format.

INTRODUCTION: Our current knowledge-based society and the many actualisations within the medical profession require a great responsibility of physicians to continuously develop and refine their skills. In this article, we reflect on some recent findings in the field of continuing education for professional doctors (continuing medical education, CME). Second, we describe the development of a CME from the Academic Center for General Practice (ACHG) of the KU Leuven. METHODS: First, we performed a literature study and we used unpublished data of a need assessment performed (2013) in a selected group of general practitioners. Second, we describe the development of a proposal to establish a CME programme for general practitioners. RESULTS: CME should go beyond the sheer acquisition of knowledge, and also seek changes in practice, attitudes and behaviours of physicians. The continuing education offerings are subject to the goals of the organising institution, but even more to the needs and desires of the end user. CONCLUSIONS: Integrated education is crucial to meet the conditions for efficient and effective continuing education. The ACHG KU Leuven decided to offer a postgraduate programme consisting of a combination of teaching methods: online courses (self-study), contact courses (traditional method) and a materials database.

Adapting a large database of point of care summarized guidelines: a process description.

RATIONALE, AIMS AND OBJECTIVES: Questions posed at the point of care (POC) can be answered using POC summarized guidelines. To implement a national POC information resource, we subscribed to a large database of POC summarized guidelines to complement locally available guidelines. Our challenge was in developing a sustainable strategy for adapting almost 1000 summarized guidelines. The aim of this paper was to describe our process for adapting a database of POC summarized guidelines. METHODS: An adaptation process based on the ADAPTE framework was tailored to be used by a heterogeneous group of participants. Guidelines were assessed on content and on applicability to the Belgian context. To improve efficiency, we chose to first aim our efforts towards those guidelines most important to primary care doctors. RESULTS: Over a period of 3 years, we screened about 80% of 1000 international summarized guidelines. For those guidelines identified as most important for primary care doctors, we noted that in about half of the cases, remarks were made concerning content. On the other hand, at least two-thirds of all screened guidelines required no changes when evaluating their local usability. CONCLUSIONS: Adapting a large body of POC summarized guidelines using a formal adaptation process is possible, even when faced with limited resources. This can be done by creating an efficient and collaborative effort and ensuring user-friendly procedures. Our experiences show that even though in most cases guidelines can be adopted without adaptations, careful review of guidelines developed in a different context remains necessary. Streamlining international efforts in adapting international POC information resources and adopting similar adaptation processes may lessen duplication efforts and prove more cost-effective.

Technology for Large-Scale Translation of Clinical Practice Guidelines: A Pilot Study of the Performance of a Hybrid Human and Computer-Assisted Approach.

BACKGROUND: The construction of EBMPracticeNet, a national electronic point-of-care information platform in Belgium, began in 2011 to optimize quality of care by promoting evidence-based decision making. The project involved, among other tasks, the translation of 940 EBM Guidelines of Duodecim Medical Publications from English into Dutch and French. Considering the scale of the translation process, it was decided to make use of computer-aided translation performed by certificated translators with limited expertise in medical translation. Our consortium used a hybrid approach, involving a human translator supported by a translation memory (using SDL Trados Studio), terminology recognition (using SDL MultiTerm terminology databases) from medical terminology databases, and support from online machine translation. This resulted in a validated translation memory, which is now in use for the translation of new and updated guidelines. OBJECTIVE: The objective of this experiment was to evaluate the performance of the hybrid human and computer-assisted approach in comparison with translation unsupported by translation memory and terminology recognition. A comparison was also made with the translation efficiency of an expert medical translator. METHODS: We conducted a pilot study in which two sets of 30 new and 30 updated guidelines were randomized to one of three groups. Comparable guidelines were translated (1) by certificated junior translators without medical specialization using the hybrid method, (2) by an experienced medical translator without this support, and (3) by the same junior translators without the support of the validated translation memory. A medical proofreader who was blinded for the translation procedure, evaluated the translated guidelines for acceptability and adequacy. Translation speed was measured by recording translation and post-editing time. The human translation edit rate was calculated as a metric to evaluate the quality of the translation. A further evaluation was made of translation acceptability and adequacy. RESULTS: The average number of words per guideline was 1195 and the mean total translation time was 100.2 minutes/1000 words. No meaningful differences were found in the translation speed for new guidelines. The translation of updated guidelines was 59 minutes/1000 words faster (95% CI 2-115; P=.044) in the computer-aided group. Revisions due to terminology accounted for one third of the overall revisions by the medical proofreader. CONCLUSIONS: Use of the hybrid human and computer-aided translation by a non-expert translator makes the translation of updates of clinical practice guidelines faster and cheaper because of the benefits of translation memory. For the translation of new guidelines, there was no apparent benefit in comparison with the efficiency of translation unsupported by translation memory (whether by an expert or non-expert translator).

Novel GMO-Based Vaccines against Tuberculosis: State of the Art and Biosafety Considerations.

Novel efficient vaccines are needed to control tuberculosis (TB), a major cause of morbidity and mortality worldwide. Several TB vaccine candidates are currently in clinical and preclinical development. They fall into two categories, the one of candidates designed as a replacement of the Bacille Calmette Guérin (BCG) to be administered to infants and the one of sub-unit vaccines designed as booster vaccines. The latter are designed as vaccines that will be administered to individuals already vaccinated with BCG (or in the future with a BCG replacement vaccine). In this review we provide up to date information on novel tuberculosis (TB) vaccines in development focusing on the risk assessment of candidates composed of genetically modified organisms (GMO) which are currently evaluated in clinical trials. Indeed, these vaccines administered to volunteers raise biosafety concerns with respect to human health and the environment that need to be assessed and managed.

Environmental risk assessment of clinical trials involving modified vaccinia virus Ankara (MVA)-based vectors.

The modified vaccinia virus Ankara (MVA) strain, which has been developed as a vaccine against smallpox, is since the nineties widely tested in clinical trials as recombinant vector for vaccination or gene therapy applications. Although MVA is renowned for its safety, several biosafety aspects need to be considered when performing the risk assessment of a recombinant MVA (rMVA). This paper presents the biosafety issues and the main lessons learned from the evaluation of the clinical trials with rMVA performed in Belgium. Factors such as the specific characteristics of the rMVA, the inserted foreign sequences/transgene, its ability for reconversion, recombination and dissemination in the population and the environment are the main points of attention. Measures to prevent or manage identified risks are also discussed.

Biosafety aspects of modified vaccinia virus Ankara (MVA)-based vectors used for gene therapy or vaccination.

The modified vaccinia virus Ankara (MVA) strain is a highly attenuated strain of vaccinia virus that has been demonstrated to be safe for humans. MVA is widely considered as the vaccinia virus strain of choice for clinical investigation because of its high safety profile. It also represents an excellent candidate for use as vector system in recombinant vaccine development for gene delivery or vaccination against infectious diseases or tumours, even in immunocompromised individuals. The use of MVA and recombinant MVA vectors must comply with various regulatory requirements, particularly relating to the assessment of potential risks for human health and the environment. The purpose of the present paper is to highlight some biological characteristics of MVA and MVA-based recombinant vectors and to discuss these from a biosafety point of view in the context of the European regulatory framework for genetically modified organisms with emphasis on the assessment of potential risks associated with environmental release.