[Is the formal quality of multiple choice questions used for continuing medical education influenced by guidelines for authors provided by the publishing houses?] / Welchen Einfluss haben Autorenrichtlinien der Verlage auf die formale Qualität von CME-Fragen in der ärztlichen Fortbildung?

INTRODUCTION: The widespread use of flawed multiple choice questions (MCQs) in continuing medical education (CME) has been demonstrated in different studies. In recent years, quality control measures have been established by some German publishing houses to ensure the quality of MCQs. OBJECTIVES: The purpose of the present study is to evaluate whether the quality of author's guidelines and the quality of MCQs correlate and whether the quality of MCQs in CME has improved following the implementation of quality control measures. MATERIALS AND METHODS: A set of 720 items from 6 journals was selected for analysis. A list of 10 item-writing flaws provided by Kühne-Eversmann et al. was modified. The existing guidelines for authors of the six selected journals were categorized by their comprehensiveness. RESULTS: There were substantial differences in item quality with the percentage of flawed items ranging between 34 % and 92 % of all items published by a journal. One journal showed considerable improvement of item quality following the implementation of guidelines for authors (61 % flawed items in 2006 vs. 33% in 2012). The comprehensiveness of the author's guidelines correlated negatively with the number of item-writing flaws. CONCLUSIONS: The correlation between the existence of elaborate guidelines for authors and the absence of item-writing flaws in MCQs suggests that the publication of guidelines for authors could result in a higher quality of MCQs, even though the review process might have a major influence.

The Learning Objective Catalogue for Patient Safety in Undergraduate Medical Education--A Position Statement of the Committee for Patient Safety and Error Management of the German Association for Medical Education.

BACKGROUND: Since the report "To err is human" was published by the Institute of Medicine in the year 2000, topics regarding patient safety and error management are in the focal point of interest of science and politics. Despite international attention, a structured and comprehensive medical education regarding these topics remains to be missing. GOALS: The Learning Objective Catalogue for Patient Safety described below the Committee for Patient Safety and Error Management of the German Association for Medical Education (GMA) has aimed to establish a common foundation for the structured implementation of patient safety curricula at the medical faculties in German-speaking countries. METHODS: The development the Learning Objective Catalogue resulted via the participation of 13 faculties in two committee meetings, two multi-day workshops, and additional judgments of external specialists. RESULTS: The Committee of Patient Safety and Error Management of GMA developed the present Learning Objective Catalogue for Patient Safety in Undergraduate Medical Education, structured in three chapters: Basics, Recognize Causes as Foundation for Proactive Behavior, and Approaches for Solutions. The learning objectives within the chapters are organized on three levels with a hierarchical organization of the topics. Overall, the Learning Objective Catalogue consists of 38 learning objectives. All learning objectives are referenced with the National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education. DISCUSSION: The Learning Objective Catalogue for Patient Safety in Undergraduate Medical Education is a product that was developed through collaboration of members from 13 medical faculties. In the German-speaking countries, the Learning Objective Catalogue should advance discussion regarding the topics of patient safety and error management and help develop subsequent educational structures. The Learning Objective Catalogue for Patient Safety can serve as a common ground for an intensified, constructive, subject-specific discussion about these topics at the medical faculties, and guide the implementation of hopefully multiple patient safety curricula in undergraduate medical education.

Enhancing diagnostic competence with self-explanation prompts and adaptable feedback.

CONTEXT: The fostering of diagnostic competence is an important goal of medical education. Worked-example cases with integrated errors represent a promising method of doing so. OBJECTIVES: This study investigates whether the provision of self-explanation prompts and adaptable feedback fosters diagnostic competence in a computer-supported learning environment based on worked-example cases with integrated errors. METHODS: A total of 103 medical students participated in the study. After a test of prior diagnostic competence, learners studied worked examples with integrated errors for cases of heart failure of different causes in a computer-based learning environment. The worked examples started with the description of a clinical situation in which a student apprentice diagnoses a patient and commits errors in the conclusions he draws from the information presented. Participants were randomly assigned to four learning conditions that varied in two factors: self-explanation prompts and adaptable feedback. In the self-explanation prompts condition, students were prompted to reflect on the errors. Independently of the condition, the student apprentice in the worked example receives feedback on his diagnosis. In the adaptable feedback condition, students were able to adjust the level of elaboration of the feedback. A post-test of diagnostic competence was administered. RESULTS: Adaptable feedback had a positive effect on important aspects of diagnostic competence (strategic knowledge: F(1,93)  = 4.15, p < 0.05, partial η(2)  = 0.04; decision-oriented practical knowledge: F(1,93)  = 4.41, p < 0.05, partial η(2)  = 0.05), whereas self-explanation prompts did not further improve the effectiveness of the learning environment. CONCLUSIONS: In the use of worked examples with integrated errors, the provision of adaptable feedback whereby learners have control over the amount of feedback they receive seems to be advantageous to the development of important decision-oriented aspects of diagnostic competence. Additional help to stimulate reflection about the error by self-explanation prompts does not seem to be critical.

Improving knowledge and changing behavior towards guideline based decisions in diabetes care: a controlled intervention study of a team-based learning approach for continuous professional development of physicians.

BACKGROUND: Continuing Professional Development (CPD) courses should ideally improve a physician's knowledge and change their professional behavior in daily practice towards a best clinical practice reference model and guideline adherence. Interactive methods such as team-based learning and case-based learning, as compared to lectures, can impart sustainable knowledge and lead to high satisfaction among participants. We designed an interactive case-based CPD-seminar on diabetes care using a team-based learning approach to evaluate whether it leads to an improvement of short-term knowledge and changing of behavior towards guideline based decisions and how this learning approach is perceived by participants. METHODS: Questionnaires and an electronic voting system were used to evaluate motivation, acceptance and knowledge of voluntary participants. Furthermore, we analyzed data on index diagnostic tests and referrals of patients with diabetes of participating physicians over a period of six months before and after the course in comparison with a matched control group in a quasi-experimental design. RESULTS: Participants (n=103) rated the interactivity and team-based discussions as the main reasons for enhanced learning. They also expected that the course would change their professional behavior. Participants scored a mean of 43.9% right answers before and 62.6% after the course (p<0.001). The referral to diabetes specialists increased by 30.8% (p<0.001). Referral for fundoscopy also increased (8.5%, n.s.) while it dropped in the control group. Furthermore, the participating physicians tested their patients more often for microalbuminuria (7.1%, n.s.). CONCLUSIONS: Our team-based learning CPD-approach was highly accepted and resulted in an increase of short-term knowledge. It significantly increased the referral to diabetes specialists in daily practice whereas all other key professional behavior indicators did change but not significantly.

Do worked examples foster medical students' diagnostic knowledge of hyperthyroidism?

OBJECTIVES: In an initial experimental study in the domain of learning about hypertension, a case-based, worked example approach was found to be most effective when erroneous examples and elaborated feedback were provided. However, combining erroneous examples with knowledge of correct result (KCR) feedback impaired learning. This study was designed to establish whether these findings could be replicated in the domain of learning about hyperthyroidism. METHODS: A total of 124 medical students were randomly assigned to the four conditions of a 2 x 2 design (with errors versus without errors; elaborated feedback versus KCR feedback). Diagnostic knowledge was operationalised by a multiple-choice test, key feature problems and problem-solving tasks. Acceptance and subjective learning outcomes were assessed on three rating scales. RESULTS: The combination of erroneous examples and elaborated feedback was the most effective learning condition, whereas erroneous examples with KCR feedback impaired knowledge acquisition. These effects were independent of differences in prior knowledge and time on task and replicated key findings of the study on hypertension diagnostics. Additionally, results showed that students in conditions with elaborated feedback assessed their learning outcomes as significantly higher than students receiving KCR feedback only. CONCLUSIONS: By providing erroneous examples in combination with elaborated feedback in a computer-based learning environment, diagnostic knowledge was fostered. The approach of 'learning from worked examples' was successfully adapted to a complex domain and was found to support the acquisition of complex competencies.

Team- and case-based learning to activate participants and enhance knowledge: an evaluation of seminars in Germany.

INTRODUCTION: There is a strong need for high-quality continuing medical education (CME) in Germany. To maintain a medical license, physicians are required to participate in regular training. Although evidence suggests that compared to lectures interactive methods can impart sustainable knowledge and a high degree of satisfaction, few interactive CME courses are offered. METHODS: We designed an interactive, team-based CME concept and launched it in a series of seminars on internal medicine. The group work was designed using team-based learning. Pre- and postcourse knowledge tests with an electronic voting system and paper questionnaires (Likert scale 1-6) were used to evaluate knowledge, motivation, and expectations of the participants (n = 159). RESULTS: Participants rated the interactivity and team-based discussion (mean: 4.57, SD: 1.41) of the CME seminars as highly important reasons to participate and stated that the CME course was very instructional (mean: 5.16, SD: 0.84) and the case discussions enhanced their learning (mean: 5.46, SD: 0.75). The majority of participants stated that their expectations were met. The participants enhanced their outcome from a mean of 47.2% right answers prior to the course to 70.3% in the postcourse test ( p < 0.001). DISCUSSION: An interactive, case-based design of a CME course following the team-based learning concept leads to a significant gain in the participants' knowledge with an identified preknowledge level. Participants' expectations seemed fulfilled by a CME course design, which combines small group discussions in the lecture hall with didactic lectures and intensive discussion.

[CME activities of medical journals: quality of multiple-choice questions as evaluation tool. Using the example of the German medical journals Deutsches Arzteblatt, Deutsche Medizinische Wochenschrift, and Der Internist]. / CME-Fortbildungsangebote in medizinischen Fachzeitschriften: Strukturqualität der MC-Fragen als Erfolgskontrollen : Am Beispiel der Fachzeitschriften Deutsches Arzteblatt, Deutsche Medizinische Wochenschrift und Der Internist.

BACKGROUND AND PURPOSE: The participation of doctors in Continuing Medical Education (CME) activities provided by medical journals increases. Therefore, their quality and efficacy need to be evaluated. METHODS: This study evaluated the quality of the multiple-choice questions (MCQs) as evaluation tool of CME activities in three German medical journals (Der Internist, Deutsches Arzteblatt, Deutsche Medizinische Wochenschrift). In total, 390 MCQs were reviewed using evidence-based principles of effective item writing. The content quality of the MCQs was not evaluated in this study. RESULTS: 264 (67.7%) of 390 MCQs contained flaws with 449 flaws in total. The proportion of flawed items was 60.8% in Deutsches Arzteblatt, 63.3% in Der Internist, and 77.9% in Deutsche Medizinische Wochenschrift. The most frequent flaws were clues (33.8%, Deutsches Arzteblatt), negatively worded stems (29.8%, Der Internist), and unfocused stems (24.7%, Deutsche Medizinische Wochenschrift). SCHLUSSFOLGERUNG: The problem of formal flaws in MCQs has an extensive relevance, as seen by the large number of studies related to this topic. The frequency of flaws (67.7%) was similar to recent studies. The distribution of the flaws suggests that with minor editing the quality of the MCQs could be substantially improved. PERSPECTIVE: The use of high-quality evaluation tools is essential to invalidate the accusation of the trivialization of CME activities, irrespective of their social context.