Evaluating student's ability to assess treatment claims: validating a German version of the Claim Evaluation Tools.

BACKGROUND: The Claim Evaluation Tools measure the ability to assess claims about treatment effects. The aim of this study was to adapt the German item sets to the target group of secondary school students (aged 11 to 16 years, grade 6 to 10) and to validate them accordingly. The scale's reliability and validity using Rasch's probabilistic test theory should be determined. METHODS: We conducted a sequential mixed-method study comprising three stages: contextualisation and adaption of the items (stage 1), piloting of the item sets using qualitative interviews (stage 2) and a construct validation by testing the unidimensional Rasch scalability for each item set after data collection in one secondary school in Germany and two secondary schools in Austria. We explored summary and individual fit statistics and performed a distractor analysis (stage 3). RESULTS: Secondary school students (n = 6) and their teachers (n = 5) participated in qualitative interviews in Germany. The qualitative interviews identified the need for minor modifications (e.g. reducing thematic repetitions, changing the order of the items). The data of 598 German and Austrian secondary school students were included to test for Rasch scalability. Rasch analyses showed acceptable overall model fit. Distractor analyses suggested that model fit could be improved by simplifying the text in the scenarios, removing and editing response options of some items. CONCLUSION: After the revision of some items, the questionnaires are suitable to evaluate secondary school students' ability to assess health claims. A future goal is to increase the pool of items being translated and tested.

A blended learning training programme for health information providers to enhance implementation of the Guideline Evidence-based Health Information: development and qualitative pilot study.

BACKGROUND: The Guideline Evidence-based Health Information was published in 2017 and addresses health information providers. The long-term goal of the guideline is to improve the quality of health information. Evidence-based health information represents a prerequisite for informed decision-making. Health information providers lack competences in evidence-based medicine. Therefore, our aim was to develop and pilot-test a blended learning training programme for health information providers to enhance application of the guideline. METHODS: 1. DEVELOPMENT: We developed the training programme according to the Medical Research Council guidance for developing and evaluating complex interventions. The training programme was planned on the basis of problem-based learning. It aims to impart competences in evidence-based medicine. Furthermore, it comprises the application of criteria for evidence-based health information. 2.Pilot testing: We conducted a qualitative pilot study focusing on the acceptability and feasibility of the training programme. Health information providers were recruited and in-house training sessions were offered. Feasibility and acceptability were explored by structured class observations and in semi-structured focus group interviews with the participants after the training sessions. The transcripts and documentations were analysed using qualitative content analysis according to Mayring. The training was revised iteratively according to the results. RESULTS: We conducted two training courses with 17 participants between November 2018 and March 2019. The adequacy of the training for the target group was identified as a major issue. There was significant heterogeneity concerning previous knowledge. Some wished to delve deeper while others seemed to be overwhelmed. In general, the work tasks were understandable. However, the participants asked for a more detailed theoretical introduction in advance. The practical relevance of the evidence-based medicine contents was rated rather low compared to the content about evidence-based health information. Based on these results, we revised the programme. CONCLUSIONS: Overall, the training proved to be feasible for implementation. Meeting the needs of all the participants was a challenge, since they were heterogeneous. Not all of them will be able or intend to implement the training contents into their working routine to the full extent. The implementation will be evaluated in a randomised controlled trial.

Enhancing Transsectoral Interdisciplinary Patient-Centered Care for Patients With Rare Cancers: Protocol for a Mixed Methods Process Evaluation.

BACKGROUND: Rare cancers account for approximately 24% of all new cancers. The category of rare tumor diseases includes almost 200 different entities. In particular, the treatment of patients with extensive care needs requires cooperation between service providers, both between sectors (outpatient and inpatient) and within sectors (eg, between different medical disciplines). The treatment pathway is associated with a high need for coordination and information sharing between providers. When crossing sectoral boundaries in the German health care system, interface problems between the outpatient and inpatient sectors can lead to gaps in care delivery. The multicomponent program Trans-sectoral Personalised Care Concept for Patients with Rare Cancers aims to optimize transsectoral cooperation and coordination of care to enhance patient involvement and the medical care coordination of patients with rare cancers. OBJECTIVE: This process evaluation will contribute to answering questions about intervention fidelity and the implementation of transsectoral communication, identifying and describing the intended and nonintended effects of the intervention, and exploring the barriers to and facilitators of the implementation. METHODS: We will include patients who participate in the intervention phase; all persons and staff involved in the development and implementation of the intervention (Onco Coach, psychologists, physicians on the contact platform, IT staff, and staff of the Bavarian Association of Statutory Health Insurance Physicians); physicians from the Ludwig-Maximilians-University Hospital Munich and the hospital of the Technical University Munich who are involved in the treatment of patients during the course of the project; and participating office-based hematologists and oncologists. Data collection will be conducted at the beginning, during, and at the end of the intervention using mixed methods. Data will be collected from questionnaires, document analyses, semistructured interviews, and structured observations and will cover different aspects of process evaluation. These include examining the context to explore existing patterns, changes in patterns, attitudes, and interactions; analyzing the implementation of intervention elements; and exploring the complex causal pathways and mediators of the intervention. Qualitative data will be analyzed using thematic analysis. The data will then be combined using between-methods triangulation. RESULTS: This project received funding on March 1, 2022. The intervention phase and recruitment for the process evaluation began on March 1, 2023, and the recruitment is expected to end on September 30, 2025. At the time of protocol submission in June 2023, a total of 8 doctors from hematology and oncology practices were enrolled. Data collection began on March 14, 2023. CONCLUSIONS: The Trans-sectoral Personalised Care Concept for Patients with Rare Cancers project is a complex intervention that is to be implemented in an equally complex health care context. The process evaluation will help understand the influence of contextual factors and assess the mechanisms of change. TRIAL REGISTRATION: ISRCTN registry ISRCTN16441179; https://doi.org/10.1186/ISRCTN16441179. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/49731.

MAPPinfo - mapping quality of health information: Validation study of an assessment instrument.

BACKGROUND: Health information is a prerequisite for informed choices-decisions, made by individuals about their own health based on knowledge and in congruence with own preferences. Criteria for development, content and design have been defined in a corresponding guideline. However, no instruments exist that provide reasonably operationalised measurement items. Therefore, we drafted the checklist, MAPPinfo, addressing the existing criteria with 19 items. OBJECTIVES: The current study aimed to validate MAPPinfo. METHODS: Five substudies were conducted subsequently at the Martin Luther University Halle-Wittenberg, Germany and the Medical University of Graz, Austria: (1) to determine content validity through expert reviews of the first draft, (2) to determine feasibility using 'think aloud' in piloting with untrained users, (3) to determine inter-rater reliability and criterion validity through a pretest on 50 health information materials, (4) to determine construct validity using 50 developers' self-declarations about development methods as a reference standard, (5) to determine divergent validity in comparison with the Ensuring Quality Information for Patients (EQIP) (expanded) Scale. The analyses used were qualitative methods and correlation-based methods for determining both inter-rater reliability and validity. RESULTS: The instrument was considered by experts to operationalise the existing guidelines convincingly. Health and nursing science students found it easy to understand and use. It also had good interrater reliability (mean of T coefficients = .79) and provided a very good estimate of the reference standard (Spearman's rho = .89), implying sound construct validity. Finally, comparison with the EQIP instrument revealed important and distinct areas of similarities and differences. CONCLUSIONS: The new instrument is ready for use as a screening instrument without the need for training. According to its underpinning concept the instrument exclusively comprises items which are justified by either ethics or research evidence, implying negligence of not yet evidence based, however, potentially important criteria. Further research is needed to complete the body of evidence-based criteria, aiming at an extension of the guideline and MAPPinfo. TRIAL REGISTRATION NUMBER: AsPredicted22546; date of registration: 24 July 2019.

Fact Boxes: What gets through? A focus group study.

BACKGROUND: The "fact box" format was developed to present the benefits and side effects of medical treatments without distortion and in a comprehensible manner. Few studies were able to show that fact boxes, when compared with other formats, significantly improve risk perception, comprehensibility and readability. The fact boxes available in Germany, however, vary with regard to how tables and accompanying texts are presented. It is unclear to what extent the existing evidence can be applied to these formats. Hardly any fact boxes have been piloted systematically. The aim was to pilot the fact box "Antibiotics for acute bronchitis" as a model for the format in terms of feasibility. METHOD: We used print and online media to recruit adults (> 18 years). The characteristics gender, age and level of education were considered for sampling. The feasibility was explored by means of a semi-structured interview guideline. The focus group interviews were audio-taped and then transcribed. Data analysis was conducted as content analysis according to Mayring. RESULTS: Five focus groups with a total of 28 participants took place between April 2018 and May 2019 (Halle/Saale, Saxony-Anhalt: n=3; St. Ingbert, Saarland: n=1; Trier, Rhineland-Palatinate: n=1). The fact box was piloted in an iterative process; the results were revised accordingly and checked again in focus groups until information saturation was achieved. The analysis identified the following main categories: interpretation of contents, readability, low reliability; credibility, relevance of the information, relevance of the presentation; superfluous information; completeness and layout. Overall, the fact box was hardly understood, especially regarding the numerical presentation given and the technical terms used. Both the objective and the intended addressees of the fact box were not clear. After revision, feasibility was established. DISCUSSION: The iterative revision process resulted in a final version that was well understood and perceived as a decision aid. CONCLUSION: The results support the demand of the guideline "Evidence-based Health Information" for piloting evidence-based health information prior to their dissemination and further evaluation. The next step will be to evaluate the fact box format in a randomised controlled trial.

Efficacy of a training programme to support the application of the guideline evidence-based health information: study protocol of a randomised controlled trial.

BACKGROUND: The evidence-based guideline entitled guideline evidence-based health information emerged from the German Network for Evidence-based Medicine (DNEbM) and was published in February 2017. The guideline addresses providers of health information and its goal is to improve the quality of health information. In addition, we explored the competences of providers of health information and developed a training programme. The aim of this study is to evaluate the efficacy of a training programme addressing providers of health information to support the application of the guideline evidence-based health information. We expected the intervention to improve the quality of health information in comparison to the provision of the guideline on its own. METHODS/DESIGN: The trial uses a superiority randomised control group design with 10 months' follow-up. Twenty-six providers of health information (groups with up to ten members) will be enrolled to compare the intervention (guideline and training programme) with usual care (a publicly available guideline). The 5-day training programme comprises an evidence-based medicine training module and a module to prepare the application of the guideline. The primary outcome parameter is the quality of the health information. Quality is operationalised as the extent of adherence to the guideline's recommendations. Each provider will prepare a single health information item informing a health-related decision on a topic freely chosen before randomisation. The quality of this information will be rated using the Mapping Health Information Quality (MAPPinfo) Checklist. An accompanying process evaluation will then be conducted. DISCUSSION: The study results should show whether the efficacy of the intervention justifies implementation of the training programme to enhance health information developers' competences in evidence-based medicine and to ensure high-quality evidence-based health information (EBHI) in the long term. TRIAL REGISTRATION: ISRCTN registry, ID: ISRCTN96941060. Registered on 7 March 2019.

Development and piloting of a blended learning training programme for physicians and medical students to enhance their competences in evidence-based decision-making.

INTRODUCTION: In 2016, the German Network for Evidence-based Medicine revised its basic curriculum for competences in evidence-based medicine. A curriculum-based training programme for physicians and medical students to enhance their competences in evidence-based decision-making was developed. The training programme was planned on the basis of problem-based learning. The aim of this qualitative pilot study was to explore the feasibility and acceptability of the training programme. Hypotheses concerning its influence on critical health literacy and the attitude toward evidence-based decision-making were to be generated. METHODS: Participating healthcare professionals received a structured training in a blended learning format. Data collection was conducted during the training sessions. The lessons were observed and protocolled and the working results were documented. Two focus group interviews were conducted after the training blocks with focus on acceptability and feasibility of the training programme. Interview transcripts and protocols were analysed using qualitative content analysis according to Mayring. Data saturation was intended by an iterative process of testing, analysing and revising the training programme. In addition, critical health literacy was assessed using the validated Critical Health Competence test. Levels of competence were calculated to measure the effect of the training on critical health competences. RESULTS: Two pilot courses with 29 physicians and other healthcare professionals were conducted between January and March 2019. Overall, the training programme proved to be feasible. The participants rated the comprehensibility of the learning modules as high. However, the practical exercises (e.g. role plays in shared decision-making) revealed that relevant subjects were insufficiently understood (e.g. the difference between the benefits and harms of a diagnostic test and its test accuracy). The interactive instructional design was appreciated. The participants appraised the work tasks as comprehensible but also challenging and requested a theoretical introduction to statistical terms in preparation for work tasks. The programme was revised iteratively according to the results. Critical health competences increased significantly after the training. Mean values (±SD) of levels of competence were 571.21 (±82.87) before training and 671.90 (±51.38) after training (p<0.0001) (levels of competence with a range from 0 to 1,000). CONCLUSION: The training programme is feasible and was well accepted by the participants. It should be established as a continuing medical education opportunity for practitioners. Evaluation in a randomised controlled trial is recommended. Furthermore, the training can easily be adapted for interprofessional training. A concept for long-term implementation is needed.

MAPPinfo, mapping quality of health information: study protocol for a validation study of an assessment instrument.

INTRODUCTION: Health information is a prerequisite of informed decision-making. Criteria for development, content and presentation have recently been published in a corresponding guideline. Within a systematic search, 27 relevant checklists were identified, none of them, however, complying with the guideline or providing reasonably operationalised measurement items. Therefore, a draft of a checklist with 19 criteria was drafted. The current study aims at developing and validating this measure of quality. METHODS AND ANALYSIS: The validation design consists of five single studies to be conducted at the University of Halle-Wittenberg/Germany and Graz/Austria. (1) Achieving content validity through expert reviews of the first draft, (2) achieving feasibility using 'think aloud' in piloting with untrained users, (3) pretesting the instrument applied to health information materials without use of secondary sources: determining inter-rater reliability and criterion validity, (4) determining construct validity using information on proceedings and methods in the development process provided by the developers and (5) determining divergent validity in comparison with the Ensuring Quality Information for Patients (EQUIP) (expanded) Scale. The substudies will use varying samples of experts, students and developers and will apply the instrument to materials of various domains. Sample sizes will be adjusted to the particular research designs and questions. Analyses will employ qualitative methods, such as content analyses and discourse within the expert panel, and correlation-based methods both for determining inter-rater reliability and validity. ETHICS AND DISSEMINATION: The project is approved by the ethics committee of the Martin Luther University Halle-Wittenberg (approval number: 2019 115). Results will be published, and the instrument made accessible on public health platforms. It is meant to become a certification standard. MAPPinfo can be used as a screening instrument without training or secondary sources. Although developed in the German language, the instrument will be applicable also in other languages. TRIAL REGISTRATION NUMBER: AsPredected22546; date of registration: 24 July 2019. PROTOCOL VERSION: July 2020.