Application of the "Plan-Do-Check-Action" plan in improving the pass rate of the "National Medical Licensing Examination".

BACKGROUND: The National Medical Licensing Examination (NMLE) is the only objective, standardized metric to evaluate whether a medical student possessing the professional knowledge and skills necessary to work as a physician. However, the overall pass rate of NMLE in our hospital in 2021 was much lower than that of Peking Union Medical College Hospital, which was required to be further improved. METHODS: To find the reasons for the unsatisfactory performance in 2021, the quality improvement team (QIT) organized regular face-to-face meetings for in-depth discussion and questionnaire, and analyzed the data by "Plato analysis" and "Brainstorming method". After finding out the reasons, the "Plan-Do-Check-Action" (PDCA) cycle was continued to identify and solve problems, which included the formulation and implementation of specific training plans by creating the "Gantt charts", the check of effects, and continuous improvements from 2021 to 2022. Detailed information about the performance of students in 2021 and 2022, and the attendance, assessment, evaluation and suggestions from our hospital were provided by the relevant departments, and the pass rate-associated data was collected online. RESULTS: After the PDCA plan, the pass rate of NMLE in our hospital increased by 10.89% from 80.15% in 2021 to 91.04% in 2022 (P = 0.0109), with the pass rate of skill examination from 95.59% in 2021 to 99.25% in 2022 (P = 0.0581) and theoretical examination from 84.5% in 2021 to 93.13% in 2022 (P = 0.027). Additionally, the mean scores of all examinees increased with the theoretical examination score increasing from 377.0 ± 98.76 in 2021 to 407.6 ± 71.94 in 2022 (P = 0.004). CONCLUSIONS: Our results showed a success application of the PDCA plan in our hospital which improved the pass rate of the NMLE in 2022, and the PDCA plan may provide a practical framework for future medical education and further improve the pass rate of NMLE in the next year.

Variance due to the examination conditions and factors associated with success in objective structured clinical examinations (OSCEs): first experiences at Paris-Saclay medical school.

BACKGROUND: We aimed to measure the variance due to examination conditions during the first sessions of objective structured clinical examinations (OSCEs) performed at a French medical school and identify factors associated with student success. METHODS: We conducted a retrospective, observational study using data from the first three OSCEs sessions performed at Paris-Saclay medical school in 2021 and 2022. For all sessions (each organized in 5 parallel circuits), we tested a circuit effect using a linear mixed-effects model adjusted for sex and the average academic level of students (according to written tests). Then, we studied the factors associated with student success at one station using a multivariate linear mixed-effects model, including the characteristics of students, assessors, and standardized patients. RESULTS: The study included three OSCEs sessions, with 122, 175, and 197 students and a mean (± SD) session score of 13.7(± 1.5)/20, 12.7(± 1.7)/20 and 12.7(± 1.9)/20, respectively. The percentage of variance due to the circuit was 6.5%, 18.2% (statistically significant), and 3.8%, respectively. For all sessions, the student's average level and station scenario were significantly associated with the score obtained in a station. Still, specific characteristics of assessors or standardized patients were only associated with the student's score in April 2021 (first session). CONCLUSION: The percentage of the variance of students' performance due to the examination conditions was significant in one out of three of the first OSCE sessions performed at Paris-Saclay medical school. This result seems more related to individual behaviors rather than specific characteristics of assessors or standardized patients, highlighting the need to continue training teaching teams. NATIONAL CLINICAL TRIAL NUMBER: Not applicable.

Evaluation de l'enseignement par les étudiants : Exemple d'un atelier de simulation sur l'ECG.

INTRODUCTION: The electrocardiogram (ECG) is a fundamental tool in medical practice. At the Faculty of Medicine of Tunis (FMT), it is usually taught during a lecture. FMT's Physiology Department has innovated its teaching by introducing simulation workshops. AIM: This study aimed to assess the students' satisfaction with teaching ECG by simulation. METHODS: This was a cross-sectional descriptive study, carried out in April 2018, including 160 students in the first year of the first cycle of medical studies, divided into 10 groups. The students attended an ECG simulation workshop at the FMT media library and then answered a satisfaction form and a self-assessment questionnaire for the workshop. RESULTS: More than 50% of the students answered either satisfied or very satisfied with the duration of the course, the room, the method of the teacher, and their participation in the course of the session. Regarding teaching support, 19.3% of the students were very satisfied with the practice of the ECG on a mannequin versus 25% for the practice on a voluntary student. For the number of students per group, 42.1% of students were dissatisfied. CONCLUSION: This study highlights the weak points of this simulation workshop in order to improve it. Then, it helps to build students' confidence and encourage their adherence to the feedback process. Finally, it shows students' enthusiasm for new teaching methods such as simulation. It would be interesting to generalize this evaluation process for the improvement of medical education and the training of future doctors.

Preclinical obesity curriculum: audit, implementation, and evaluation.

BACKGROUND: This study aimed to (1) evaluate the current status of obesity education at Case Western Reserve University School of Medicine (CWRU) (2), introduce a comprehensive first-year curriculum on obesity, and (3) assess the impact of the curriculum on self-reported attitudes and knowledge regarding obesity among first-year medical students. METHODS: The preclinical curriculum at CWRU was reviewed to determine the degree of coverage of Obesity Medicine Education Collaborative (OMEC) competencies for healthcare professionals, and recommendations were provided for revising the curriculum to better adhere to these evidence-based competencies. A survey on obesity attitudes and knowledge was given before and after the implementation of the new curriculum to measure intervention-related changes. Changes in obesity attitudes and knowledge were compared (1) before and after the intervention for the class of 2025 and (2) after the intervention for the class of 2025 to a historical cohort that did not receive the intervention. RESULTS: Among the 27 competencies examined in the audit, 55% were unmet and 41% were partially met. Of 186 first-year medical students (M1s), 29 (16%) completed the baseline survey and 26 (14%) completed the post-intervention survey. Following the intervention, there was a notable improvement in attitudes and knowledge regarding obesity. Specifically, there was a significant decrease in the belief that obesity is caused by poor personal choices, and knowledge of obesity in fourteen out of fifteen areas showed significant improvement from pre- to post-intervention. Additionally, obesity attitudes and knowledge were significantly better post-intervention when compared to the historical cohort. CONCLUSIONS: The improvements made to the preclinical curriculum through this project improved obesity attitudes and knowledge among first-year medical students. This method provides a practical approach for evaluating and enhancing obesity education in medical school curricula.

Frontline assessors' opinions about grading committees in a medicine clerkship.

BACKGROUND: Collective decision-making by grading committees has been proposed as a strategy to improve the fairness and consistency of grading and summative assessment compared to individual evaluations. In the 2020-2021 academic year, Washington University School of Medicine in St. Louis (WUSM) instituted grading committees in the assessment of third-year medical students on core clerkships, including the Internal Medicine clerkship. We explored how frontline assessors perceive the role of grading committees in the Internal Medicine core clerkship at WUSM and sought to identify challenges that could be addressed in assessor development initiatives. METHODS: We conducted four semi-structured focus group interviews with resident (n = 6) and faculty (n = 17) volunteers from inpatient and outpatient Internal Medicine clerkship rotations. Transcripts were analyzed using thematic analysis. RESULTS: Participants felt that the transition to a grading committee had benefits and drawbacks for both assessors and students. Grading committees were thought to improve grading fairness and reduce pressure on assessors. However, some participants perceived a loss of responsibility in students' grading. Furthermore, assessors recognized persistent challenges in communicating students' performance via assessment forms and misunderstandings about the new grading process. Interviewees identified a need for more training in formal assessment; however, there was no universally preferred training modality. CONCLUSIONS: Frontline assessors view the switch from individual graders to a grading committee as beneficial due to a perceived reduction of bias and improvement in grading fairness; however, they report ongoing challenges in the utilization of assessment tools and incomplete understanding of the grading and assessment process.

Palliative care education in undergraduate medical and nursing programs in Colombia: a cross-sectional analysis.

BACKGROUND: The number of people suffering from chronic diseases requiring palliative care (PC) is increasing rapidly. Therefore, PC teaching in undergraduate health science programs is necessary to improve primary PC based on international recommendations and available scientific evidence. METHODS: A descriptive cross-sectional study was conducted. Active undergraduate medical and nursing programs that were approved by the Colombian Ministry of Education and integrated PC teaching into their curricula were included in the study. The total sample consisted of 48 programs: 31 nursing and 17 medical programs. RESULTS: PC competencies are distributed throughout the curriculum in 41.67% of programs, in elective courses in 31.25%, and in mandatory courses in 27.08% of the programs. The average PC teaching hours is 81 for nursing and 57.6 for medicine. PC clinical rotations are not offered in 75% of the programs. For undergraduate nursing programs, the most frequent competencies taught are the definition and history of PC and identifying common symptoms associated with advanced disease. In undergraduate medicine, the most common competencies are pharmacological and non-pharmacological pain management and identification of PC needs. CONCLUSIONS: PC teaching in undergraduate health science programs mainly addresses the conceptual and theoretical aspects of PC, which are part of the competencies present throughout the programs' curricula. Low availability of PC clinical rotations was identified. Future studies should assess whether the low availability of clinical rotations in PC limits the ability of students to develop the practical competencies necessary to provide quality PC. TRIAL REGISTRATION: Not applicable.

Clues for improvement of research in objective structured clinical examination.

While objective clinical structured examination (OSCE) is a worldwide recognized and effective method to assess clinical skills of undergraduate medical students, the latest Ottawa conference on the assessment of competences raised vigorous debates regarding the future and innovations of OSCE. This study aimed to provide a comprehensive view of the global research activity on OSCE over the past decades and to identify clues for its improvement. We performed a bibliometric and scientometric analysis of OSCE papers published until March 2024. We included a description of the overall scientific productivity, as well as an unsupervised analysis of the main topics and the international scientific collaborations. A total of 3,224 items were identified from the Scopus database. There was a sudden spike in publications, especially related to virtual/remote OSCE, from 2020 to 2024. We identified leading journals and countries in terms of number of publications and citations. A co-occurrence term network identified three main clusters corresponding to different topics of research in OSCE. Two connected clusters related to OSCE performance and reliability, and a third cluster on student's experience, mental health (anxiety), and perception with few connections to the two previous clusters. Finally, the United States, the United Kingdom, and Canada were identified as leading countries in terms of scientific publications and collaborations in an international scientific network involving other European countries (the Netherlands, Belgium, Italy) as well as Saudi Arabia and Australia, and revealed the lack of important collaboration with Asian countries. Various avenues for improving OSCE research have been identified: i) developing remote OSCE with comparative studies between live and remote OSCE and issuing international recommendations for sharing remote OSCE between universities and countries; ii) fostering international collaborative studies with the support of key collaborating countries; iii) investigating the relationships between student performance and anxiety.

Evaluating competency-based medical education: a systematized review of current practices.

BACKGROUND: Few published articles provide a comprehensive overview of the available evidence on the topic of evaluating competency-based medical education (CBME) curricula. The purpose of this review is therefore to synthesize the available evidence on the evaluation practices for competency-based curricula employed in schools and programs for undergraduate and postgraduate health professionals. METHOD: This systematized review was conducted following the systematic reviews approach with minor modifications to synthesize the findings of published studies that examined the evaluation of CBME undergraduate and postgraduate programs for health professionals. RESULTS: Thirty-eight articles met the inclusion criteria and reported evaluation practices in CBME curricula from various countries and regions worldwide, such as Canada, China, Turkey, and West Africa. 57% of the evaluated programs were at the postgraduate level, and 71% were in the field of medicine. The results revealed variation in reporting evaluation practices, with numerous studies failing to clarify evaluations' objectives, approaches, tools, and standards as well as how evaluations were reported and communicated. It was noted that questionnaires were the primary tool employed for evaluating programs, often combined with interviews or focus groups. Furthermore, the utilized evaluation standards considered the well-known competencies framework, specialized association guidelines, and accreditation criteria. CONCLUSION: This review calls attention to the importance of ensuring that reports of evaluation experiences include certain essential elements of evaluation to better inform theory and practice.

Sexual and gender minority content in undergraduate medical education in the United States and Canada: current state and changes since 2011.

PURPOSE: To characterize current lesbian, gay, bisexual, transgender, queer, and intersex (LGBTQI +) health-related undergraduate medical education (UME) curricular content and associated changes since a 2011 study and to determine the frequency and extent of institutional instruction in 17 LGBTQI + health-related topics, strategies for increasing LGBTQI + health-related content, and faculty development opportunities. METHOD: Deans of medical education (or equivalent) at 214 allopathic or osteopathic medical schools in Canada and the United States were invited to complete a 36-question, Web-based questionnaire between June 2021 and September 2022. The main outcome measured was reported hours of LGBTQI + health-related curricular content. RESULTS: Of 214 schools, 100 (46.7%) responded, of which 85 (85.0%) fully completed the questionnaire. Compared to 5 median hours dedicated to LGBTQI + health-related in a 2011 study, the 2022 median reported time was 11 h (interquartile range [IQR], 6-16 h, p < 0.0001). Two UME institutions (2.4%; 95% CI, 0.0%-5.8%) reported 0 h during the pre-clerkship phase; 21 institutions (24.7%; CI, 15.5%-33.9%) reported 0 h during the clerkship phase; and 1 institution (1.2%; CI, 0%-3.5%) reported 0 h across the curriculum. Median US allopathic clerkship hours were significantly different from US osteopathic clerkship hours (4 h [IQR, 1-6 h] versus 0 h [IQR, 0-0 h]; p = 0.01). Suggested strategies to increase content included more curricular material focusing on LGBTQI + health and health disparities at 55 schools (64.7%; CI, 54.6%-74.9%), more faculty willing and able to teach LGBTQI + -related content at 49 schools (57.7%; CI, 47.1%-68.2%), and more evidence-based research on LGBTQI + health and health disparities at 24 schools (28.2%; CI, 18.7%-37.8%). CONCLUSION: Compared to a 2011 study, the median reported time dedicated to LGBTQI + health-related topics in 2022 increased across US and Canadian UME institutions, but the breadth, efficacy, or quality of instruction continued to vary substantially. Despite the increased hours, this still falls short of the number of hours based on recommended LGBTQI + health competencies from the Association of American Medical Colleges. While most deans of medical education reported their institutions' coverage of LGBTQI + health as 'fair,' 'good,' or 'very good,' there continues to be a call from UME leadership to increase curricular content. This requires dedicated training for faculty and students.

Gender differences in learning and study strategies impact medical students' preclinical and USMLE step 1 examination performance.

BACKGROUND: Evaluation of students' learning strategies can enhance academic support. Few studies have investigated differences in learning strategies between male and female students as well as their impact on United States Medical Licensing Examination® (USMLE) Step 1 and preclinical performance. METHODS: The Learning and Study Strategies Inventory (LASSI) was administered to the classes of 2019-2024 (female (n = 350) and male (n = 262)). Students' performance on preclinical first-year (M1) courses, preclinical second-year (M2) courses, and USMLE Step 1 was recorded. An independent t-test evaluated differences between females and males on each LASSI scale. A Pearson product moment correlation determined which LASSI scales correlated with preclinical performance and USMLE Step 1 examinations. RESULTS: Of the 10 LASSI scales, Anxiety, Attention, Information Processing, Selecting Main Idea, Test Strategies and Using Academic Resources showed significant differences between genders. Females reported higher levels of Anxiety (p < 0.001), which significantly influenced their performance. While males and females scored similarly in Concentration, Motivation, and Time Management, these scales were significant predictors of performance variation in females. Test Strategies was the largest contributor to performance variation for all students, regardless of gender. CONCLUSION: Gender differences in learning influence performance on STEP1. Consideration of this study's results will allow for targeted interventions for academic success.

Position statement of the GMA committee "teaching evaluation".

The evaluation of teaching can be an essential driver for curriculum development. Instruments for teaching evaluation are not only used for the purpose of quality assurance but also in the context of medical education research. Therefore, they must meet the common requirements for reliability and validity. This position paper from the GMA Teaching Evaluation Committee discusses strategic and methodological aspects of evaluation in the context of undergraduate medical education and related courses; and formulates recommendations for the further development of evaluation. First, a four-step approach to the design and implementation of evaluations is presented, then methodological and practical aspects are discussed in more detail. The focus here is on target and confounding variables, survey instruments as well as aspects of implementation and data protection. Finally, possible consequences from evaluation data for the four dimensions of teaching quality (structural and procedural aspects, teachers and outcomes) are discussed.

Scientific competence during medical education - insights from a cross-sectional study at a German Medical School.

BACKGROUND: Medical knowledge regarding the pathophysiology, diagnosis and treatment of diseases is constantly evolving. To effectively incorporate these findings into professional practice, it is crucial that scientific competencies are a central component of medical education. This study seeks to analyse the current state of scientific education and students' desires for integration into the curriculum. METHODS: From October to December 2022, a survey was distributed at the Medical Faculty Dresden to all medical students from the 1st to 5th academic year (AY). The survey investigates current expectations of applying scientific competencies later in professional life, and the students were asked to self-assess various scientific skills and in relation to the National Competence Based Catalogue of Learning Objectives for Undergraduate Medical Education. The self-assessments were objectified through a competence test with ten multiple-choice questions. The desire for curricular teaching was inquired. RESULTS: 860 students completed the survey. This corresponds to a response rate of 64%. In the 5th AY, approximately 80% of the participants stated that they expected to work with scientific literature on a daily to monthly basis in future professional life and to communicate corresponding scientific findings to patients. Only 30-40% of the 5th AY rate their scientific competencies as sufficient to do this appropriately. This corresponds with the self-assessed competencies that only slightly increased over the 5 AYs from 14.1 ± 11.7 to 21.3 ± 13.8 points (max. 52) and is also reflected in the competence test (1st AY 3.6 ± 1.75 vs. 5th AY 5.5 ± 1.68, max. 10 points). Half of the students in the 4th and 5th AYs were dissatisfied with the current teaching of scientific skills. The majority preferred the implementation of a science curriculum (56%), preferably as seminars dealing with topics such as literature research, analysis, and science communication. CONCLUSIONS: The results show discrepancies between expectations of using scientific knowledge in everyday professional life, self-rated and objectively recorded competencies, and the current state of curricular teaching of scientific competencies. There is a strong need for adequate practical training, particularly in critical analyses of scientific literature, which enables the communication of scientific knowledge to patients.

Designing an evaluation tool for evaluating training programs of medical students in clinical skill training center from consumers' perspective.

INTRODUCTION: The Clinical Skill Training Center (CSTC) is the first environment where third year medical students learn clinical skills after passing basic science. Consumer- based evaluation is one of the ways to improve this center with the consumer. This study was conducted with the aim of preparing a consumer-oriented evaluation tool for CSTC among medical students. METHOD: The study was mixed method. The first phase was qualitative and for providing an evaluation tool. The second phase was for evaluating the tool. At the first phase, after literature review in the Divergent phase, a complete list of problems in the field of CSTC in medicine schools was prepared. In the convergent step, the prepared list was compared with the standards of clinical education and values of scriven. In the second phase it was evaluated by the scientific and authority committee. Validity has been measured by determining CVR and CVI: Index. The face and content validity of the tool was obtained through the approval of a group of specialists. RESULTS: The findings of the research were in the form of 4 questionnaires: clinical instructors, pre-clinical medical students, and interns. All items were designed as a 5-point Likert. The main areas of evaluation included the objectives and content of training courses, implementation of operations, facilities and equipment, and the environment and indoor space. In order to examine the long-term effects, a special evaluation form was designed for intern. CONCLUSION: The tool for consumer evaluation was designed with good reliability and trustworthiness and suitable for use in the CSTC, and its use can improve the effectiveness of clinical education activities.

Effectiveness of the Introduction to Critical Care in Emergency Medicine Curriculum's Implementation Among Trainees Interested in Intensive Care.

BACKGROUND: Emergency physicians and trainees provide the initial care for critically ill patients. In times of emergency department boarding, this care may extend beyond the first few hours. To meet the needs of this population, a standardized novel critical care curriculum targeting third- and fourth-year medical students was developed. OBJECTIVES: We hypothesized that the institution of such a curriculum is feasible and will provide an increased understanding of the underlying critical care principles within this learner population. METHODS: We developed a 2-month-long critical care curriculum (February-April) and carried out the course twice from 2022-2023. Our pilot study deployed this curriculum to medical students interested in critical care through the American Academy of Emergency Medicine/Resident and Student Association. The primary outcome included was the overall composite score comparison of the pre- and post-course evaluations, with a higher score indicating that the student improved their comprehension. Secondary outcomes included the individual factors of the pre- and post-course surveys. RESULTS: Fifty-one trainees completed the pilot course, including 11/51 (21.6%) third-year medical students and 40/51 (78.4%) fourth-year medical students. Overall, 39 had "no experience" in critical care and 12 indicated that they had "previous experience." The students' baseline pre-course from the pooled 2022 and 2023 Introduction to Critical Care in Emergency Medicine (ICCEM) curriculum data was 3 (interquartile range 4-3) and their post-course score was 9 (interquartile range 9-9), p-value 0.015 for the 51/54 students who completed the course. CONCLUSIONS: The novel curriculum was found to be effective during its implementation in third- and fourth-year medical students. As such, it indicated that a critical care fundamentals course improves confidence in these topics for students with and without prior experience. Further work is necessary to understand the generalizability and knowledge retention of the proposed pilot curriculum.

The Military Health History: An Educational Initiative to Improve Veteran Healthcare.

INTRODUCTION: As a result of the nature of military service, veterans are a unique patient population with many special health considerations. For various reasons, measures are often not taken by clinicians to address such special considerations. This results in a healthcare disparity for veterans first described by Dr. Jeffrey Brown in 2012. To address this disparity, we introduced "the military health history" to third-year medical students at a large medical school in the southeastern United States. Our objective was to assess the effectiveness of this educational intervention and determine its potential role in creating a future in which veteran healthcare is of the highest quality. MATERIALS AND METHODS: This study was approved as a quality assurance/quality improvement project by both Louisiana State University Health Science Center and Southeast Louisiana Veteran Healthcare System IRB offices. A short lecture outlining the components of the military health history was presented to 186 third-year medical students. The students were given the opportunity to answer five survey questions before and after the lecture. These questions assessed the students' current confidence performing a military health history, perceived importance of doing so, and likelihood of future implementation. To determine useful retention of the lecture material, the post-lecture survey was readministered to the same population sample 6 months after the lecture. A series of repeated measures analyses of variance were conducted to examine changes in mean levels of confidence, importance, and likelihood of ascertaining military history during a patient encounter at pre- and post-presentation as well as at the 6 month follow-up. RESULTS: Results revealed a significant effect of time on importance, Wilks' Lambda = 0.74, F (2, 87) = 15.41, P < 0.001; confidence, Wilks' Lambda = 0.61, F (2, 87) = 27.58, P < 0.001; and likelihood of ascertaining a military history during a future patient encounter, Wilks' Lambda = 0.46, F (2, 88) = 50.58, P < 0.001. Results are demonstrated in detail in Table I of the manuscript. CONCLUSION: The lecture resulted in a statistically significant increase over 6 months in both the likelihood and confidence parameters. The team believes that this result indicates that the students demonstrated useful retention of the lecture material. Our hope is that these students continue to employ the military health history throughout their years of clinical work. In the future, we plan to survey veterans immediately following Veterans Health Administration clinic visits with members of our study population to assess the patient's perceived benefit of the military health history. The team will continue to investigate ways in which military health curricula can be implemented in undergraduate medical education.

MD-Granting Medical Schools in the US, 2022-2023.

This Appendix presents data derived from the 2022-2023 Liaison Committee on Medical Education Annual Medical School Questionnaire-Part II.

341º Reunião Ordinária do Conselho Nacional de Saúde - 13 abr. 2023