Evidence of learning in workplace-based assessments in a Family Medicine Training Programme.

BACKGROUND:  Learning portfolios (LPs) provide evidence of workplace-based assessments (WPBAs) in clinical settings. The educational impact of LPs has been explored in high-income countries, but the use of portfolios and the types of assessments used for and of learning have not been adequately researched in sub-Saharan Africa. This study investigated the evidence of learning in registrars' LPs and the influence of the training district and year of training on assessments. METHODS:  A cross-sectional study evaluated 18 Family Medicine registrars' portfolios from study years 1-3 across five decentralised training sites affiliated with the University of the Witwatersrand. Descriptive statistics were calculated for the portfolio and quarterly assessment (QA) scores and self-reported clinical skills competence levels. The competence levels obtained from the portfolios and university records served as proxy measures for registrars' knowledge and skills. RESULTS:  The total LP median scores ranged from 59.9 to 81.0, and QAs median scores from 61.4 to 67.3 across training years. The total LP median scores ranged from 62.1 to 83.5 and 62.0 to 67.5, respectively in QAs across training districts. Registrars' competence levels across skill sets did not meet the required standards. Higher skills competence levels were reported in the women's health, child health, emergency care, clinical administration and teaching and learning domains. CONCLUSION:  The training district and training year influence workplace-based assessment (WPBA) effectiveness. Ongoing faculty development and registrar support are essential for WPBA.Contribution: This study contributes to the ongoing discussion of how to utilise WPBA in resource-constrained sub-Saharan settings.

How Do Students Critically Evaluate Outdated Language That Relates to Gender in Biology?

Cisheteronormative ideologies are infused into every aspect of society, including undergraduate science. We set out to identify the extent to which students can identify cisheteronormative language in biology textbooks by posing several hypothetical textbook questions and asking students to modify them to make the language more accurate (defined as "correct; precise; using language that applies to all people"). First, we confirmed that textbooks commonly use language that conflates or confuses sex and gender. We used this information to design two sample questions that used similar language. We examined what parts of the questions students modified, and the changes they recommended. When asked to modify sample textbook questions, we found the most common terms or words that students identified as inaccurate were related to infant gender identity. The most common modifications that students made were changing gender terms to sex terms. Students' decisions in this exercise differed little across three large biology courses or by exam performance. As the science community strives to promote inclusive classrooms and embrace the complexity of human gender identities, we provide foundational information about students' ability to notice and correct inaccurate language related to sex and gender in biology.

Designing an evaluation tool for evaluating training programs of medical students in clinical skill training center from consumers' perspective.

INTRODUCTION: The Clinical Skill Training Center (CSTC) is the first environment where third year medical students learn clinical skills after passing basic science. Consumer- based evaluation is one of the ways to improve this center with the consumer. This study was conducted with the aim of preparing a consumer-oriented evaluation tool for CSTC among medical students. METHOD: The study was mixed method. The first phase was qualitative and for providing an evaluation tool. The second phase was for evaluating the tool. At the first phase, after literature review in the Divergent phase, a complete list of problems in the field of CSTC in medicine schools was prepared. In the convergent step, the prepared list was compared with the standards of clinical education and values of scriven. In the second phase it was evaluated by the scientific and authority committee. Validity has been measured by determining CVR and CVI: Index. The face and content validity of the tool was obtained through the approval of a group of specialists. RESULTS: The findings of the research were in the form of 4 questionnaires: clinical instructors, pre-clinical medical students, and interns. All items were designed as a 5-point Likert. The main areas of evaluation included the objectives and content of training courses, implementation of operations, facilities and equipment, and the environment and indoor space. In order to examine the long-term effects, a special evaluation form was designed for intern. CONCLUSION: The tool for consumer evaluation was designed with good reliability and trustworthiness and suitable for use in the CSTC, and its use can improve the effectiveness of clinical education activities.

The effect of scenario-based training versus video training on nurse anesthesia students' basic life support knowledge and skill of cardiopulmonary resuscitation: a quasi-experimental comparative study.

BACKGROUND: Performing CPR (Cardiopulmonary Resuscitation) is an extremely intricate skill whose success depends largely on the level of knowledge and skill of Anesthesiology students. Therefore, this research was conducted to compare the effect of the scenario-based training method as opposed to video training method on nurse anesthesia students' BLS (Basic Life Support) knowledge and skills. METHODS: This randomized quasi-experimental study involved 45 nurse anesthesia students of Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran in 2022-2023. The practical room of the university formed the research environment. The participants were randomly divided into three groups of scenario-based training, video training, and control. Data were collected by a knowledge questionnaire and a BLS skill assessment checklist before and after the intervention. RESULTS: There was a significant difference between the students' scores of BLS knowledge and skill before and after the educational intervention in both SG (scenario group) (p < 0.001) and VG (video group) (p = 0.008) (p < 0.001). However, no significant difference was observed in this regard in the CG (control group) (p = 0.37) (p = 0.16). Also, the mean scores of BLS knowledge and skills in the SG were higher than those in the VG (p < 0.001). CONCLUSION: Given the beneficial impact of scenario-based education on fostering active participation, critical thinking, utilization of intellectual abilities, and learner creativity, it appears that this approach holds an advantage over video training, particularly when it comes to teaching crucial subjects like Basic Life Support.

Role of artificial intelligence (Google bard) in morphological, histopathological, and radiological image identifications: Objective Structured Practical Examination (OSPE) type-based performance.

OBJECTIVES: To evaluate the role of artificial intelligence (Google Bard) in figures, scans, and image identifications and interpretations in medical education and healthcare sciences through an Objective Structured Practical Examination (OSPE) type of performance. METHODS: The OSPE type of question bank was created with a pool of medical sciences figures, scans, and images. For assessment, 60 figures, scans and images were selected and entered into the given area of the Google Bard to evaluate the knowledge level. RESULTS: The marks obtained by Google Bard in brain structures, morphological and radiological images 7/10 (70%); bone structures, radiological images 9/10 (90%); liver structure and morphological, pathological images 4/10 (40%); kidneys structure and morphological images 2/7 (28.57%); neuro-radiological images 4/7 (57.14%); and endocrine glands including the thyroid, pancreas, breast morphological and radiological images 8/16 (50%). The overall total marks obtained by Google Bard in various OSPE figures, scans, and image identification questions were 34/60 (56.7%). CONCLUSION: Google Bard scored satisfactorily in morphological, histopathological, and radiological image identifications and their interpretations. Google Bard may assist medical students, faculty in medical education and physicians in healthcare settings.

The Effect of Distance Reiki on State Test Anxiety and Test Performance: A Randomized Controlled Trial of Nursing Students.

BACKGROUND: This randomized controlled, pretest-post-test intervention study examined the effect of distance reiki on state test anxiety and test performance. METHOD: First-year nursing students (n = 71) were randomized into two groups. One week before the examination, intervention group participants performed reiki remotely for 20 minutes for 4 consecutive days, and control group participants received no intervention. RESULTS: The intervention group had lower posttest cognitive and psychosocial subscale scores than pretest scores (p > .05). The control group had a significantly higher mean posttest physiological subscale score than pretest score (p < .05). Final grade point averages were not significantly different between the intervention and control groups (p > .05). One quarter of the intervention group participants noted reiki reduced their stress and helped them perform better on the examination. CONCLUSION: Reiki is a safe and easy-to-practice method to help students cope with test anxiety. [J Nurs Educ. 2024;63(5):298-303.].

Determining the psychometric properties of a written test to assess safe dental practice.

INTRODUCTION: Safe practice in medicine and dentistry has been a global priority area in which large knowledge gaps are present.Patient safety strategies aim at preventing unintended damage to patients that can be caused by healthcare practitioners. One of the components of patient safety is safe clinical practice. Patient safety efforts will help in ensuring safe dental practice for early detection and limiting non-preventable errors.A valid and reliable instrument is required to assess the knowledge of dental students regarding patient safety. OBJECTIVE: To determine the psychometric properties of a written test to assess safe dental practice in undergraduate dental students. MATERIAL AND METHODS: A test comprising 42 multiple-choice questions of one-best type was administered to final year students (52) of a private dental college. Items were developed according to National Board of Medical Examiners item writing guidelines. The content of the test was determined in consultation with dental experts (either professor or associate professor). These experts had to assess each item on the test for language clarity as A: clear, B: ambiguous and relevance as 1: essential, 2: useful, not necessary, 3: not essential. Ethical approval was taken from the concerned dental college. Statistical analysis was done in SPSS V.25 in which descriptive analysis, item analysis and Cronbach's alpha were measured. RESULT: The test scores had a reliability (calculated by Cronbach's alpha) of 0.722 before and 0.855 after removing 15 items. CONCLUSION: A reliable and valid test was developed which will help to assess the knowledge of dental students regarding safe dental practice. This can guide medical educationist to develop or improve patient safety curriculum to ensure safe dental practice.

Factors associated with infectious diseases fellowship academic success.

BACKGROUND: A multitude of factors are considered in an infectious disease (ID) training program's meticulous selection process of ID fellows but their correlation to pre and in-fellowship academic success as well as post-fellowship academic success and short-term outcomes is poorly understood. Our goal was to investigate factors associated with subsequent academic success in fellowship as well as post-fellowship short-term outcomes. METHODS: In 2022, we retrospectively analyzed deidentified academic records from 39 graduates of the Mayo Clinic Rochester ID Fellowship Program (1 July 2013- 30 June 2022). Data abstracted included demographics, degrees, honor society membership, visa/citizenship status, medical school, residency training program, United States Medical Licensure Exam (USMLE) scores, letters of recommendation, in-training examination (ITE) scores, fellowship track, academic rank, career choice, number of honors, awards, and abstracts/publications prior to fellowship, during training, and within 2 years of graduation. RESULTS: Younger fellows had higher USMLE step 1 scores, pre and in-fellowship scholarly productivity, and higher ITE performance. Female fellows had significantly higher USMLE step 3 scores. Prior research experience translated to greater in-fellowship scholarly productivity. Higher USMLE scores were associated with higher ID ITE performance during multiple years of fellowship, but USMLE step 2 clinical knowledge and 3 scores were associated with higher pre and in-fellowship scholarly productivity and receiving an award during fellowship. The USMLE step 1 score did not correlate with fellowship performance beyond year 1 and 2 ITE scores. CONCLUSIONS: Multiple aspects of a prospective fellow's application must be considered as part of a holistic review process for fellowship selection. USMLE step 2 CK and 3 scores may predict fellowship performance across multiple domains.

Effectiveness of digital and analog learning methods for learning anatomical structures in physiotherapy education.

BACKGROUND: According to the German Physiotherapy Education and Qualification Regulations, teaching of anatomical structures is one of the fundamental subjects of physiotherapy education. Besides exhibits and models, anatomy atlases are usually used as teaching and learning tools. These are available in both analog form such as printed books or in digital form as a mobile application. Furthermore, the use of digital teaching and learning tools is steadily increasing within the education of health professionals. AIM: To assess the efficacy of a digital educational tool in contrast to an analog anatomical atlas in acquiring knowledge about anatomical structures. MATERIAL AND METHOD: The data collection took place in the context of an anatomy tutorial for students of the bachelor's degree program in physiotherapy. In a cross-over design, the students completed two learning assignments, each, with different learning materials provided, either with an anatomy app on a tablet or with an anatomy atlas as a book. The tests to assess the newly acquired knowledge immediately after the task, consisted of questions about the anatomical structures of the knee as well as the shoulder. In addition, the students' satisfaction with the learning materials provided was surveyed using a questionnaire. The survey assessed their satisfaction, their assessment of learning success, and their affinity to digital learning materials. This was done using a 5-point Likert scale and a free-text field. The data was analyzed descriptively, and group differences were calculated using a t-tests. RESULTS: Thirty students participated. The group comparison showed a significantly better outcome for the group that prepared with the analog anatomy atlas for the questions on the knee than the comparison group that used the anatomy app (t(28) = 2.6; p = 0.007). For the questions concerning the shoulder, there was no significant difference between the digital and analog groups (t(28) = 1.14; p = 0.26). The questionnaire revealed that satisfaction with the analog anatomy atlas was significantly higher than with the anatomy app. A total of 93.34% rated their experience with the analog learning tool at least "somewhat satisfied". In contrast, 72.67% of students partially or fully agreed that they "enjoyed learning with digital learning tools". DISCUSSION: Learning anatomical structures with the Human Anatomy Atlas 2023 + app did not show a clear advantage when compared to an anatomy book in these two cohorts of physiotherapy students. The results of the questionnaire also showed greater satisfaction with the analog anatomy atlas than with the anatomy app, whereas most students stated that they frequently use digital learning tools, including some for anatomical structures. Satisfaction with the learning tool seems to play a central role in their effectiveness. In addition, sufficient time must be provided for users to familiarize themselves with the user interface of digital applications to use them effectively. REGISTRATION: Diese klinische Studie wurde nicht in einem Studienregister registriert.

The reliability of the College of Intensive Care Medicine of Australia and New Zealand "Hot Case" examination.

BACKGROUND: High stakes examinations used to credential trainees for independent specialist practice should be evaluated periodically to ensure defensible decisions are made. This study aims to quantify the College of Intensive Care Medicine of Australia and New Zealand (CICM) Hot Case reliability coefficient and evaluate contributions to variance from candidates, cases and examiners. METHODS: This retrospective, de-identified analysis of CICM examination data used descriptive statistics and generalisability theory to evaluate the reliability of the Hot Case examination component. Decision studies were used to project generalisability coefficients for alternate examination designs. RESULTS: Examination results from 2019 to 2022 included 592 Hot Cases, totalling 1184 individual examiner scores. The mean examiner Hot Case score was 5.17 (standard deviation 1.65). The correlation between candidates' two Hot Case scores was low (0.30). The overall reliability coefficient for the Hot Case component consisting of two cases observed by two separate pairs of examiners was 0.42. Sources of variance included candidate proficiency (25%), case difficulty and case specificity (63.4%), examiner stringency (3.5%) and other error (8.2%). To achieve a reliability coefficient of > 0.8 a candidate would need to perform 11 Hot Cases observed by two examiners. CONCLUSION: The reliability coefficient for the Hot Case component of the CICM second part examination is below the generally accepted value for a high stakes examination. Modifications to case selection and introduction of a clear scoring rubric to mitigate the effects of variation in case difficulty may be helpful. Increasing the number of cases and overall assessment time appears to be the best way to increase the overall reliability. Further research is required to assess the combined reliability of the Hot Case and viva components.

Contribution of ethical reasoning learning sessions on medical training.

INTRODUCTION: Ethical reasoning is an important skill for all physicians who often face complex ethical dilemmas in their daily practice. Therefore, medical training should include methods for learning ethical theories and concepts, as well as how to apply them in practical situations. AIM: Assess the contribution of an Ethical Reasoning Learning session to fifth medical students' training through a comparison of results of the same objective and structured clinical examination (OSCE) in the form of simulated interview before and after sessions. METHODS: Four 45- minutes' sessions of Ethical Reasoning Learning (ERL) were implemented during a psychiatry internship for four groups of 5th-year students of the faculty of medicine of Monastir (Tunisia). Each session was divided into 7 parts: introduction, reading of a clinical vignette, brainstorming concerning the problems posed by this clinical situation, classification of the problems, identification of the principles of medical ethics, construction of the ethical matrix, and a conclusion. RESULTS: Fifty-seven students participated in the study divided into 4 groups. We found a significant difference in the means of the OSCE scores before and after the ERL session and a significant difference between the probability of respecting medical secrecy during pre and post-ethical reasoning learning sessions (p <0.001). We have found an effect of ERL sessions on the acquisition of this ethical competence by medical students. CONCLUSION: We learned that an ERL session has improved medical training in ethics applied to psychiatry. Other sessions dealing with other ethical skills are necessary to confirm these results.

Patient-management problem in the evaluation of intensive medicine.

INTRODUCTION: In intensive care medicine (ICM), the use of Patient-Management Problem (PMP) remains limited and no feedback from students is available. AIM: To compare the feasibility of employing PMP referring to clinical cases (CC) as assessment tools for appraising the knowledge and competencies in ICM students; and to gather the students' perception regarding this experience. METHODS: it was a cross-sectional randomized trial. Were included, external students in the 3rd year of the 2nd cycle of medical studies (3rd-SCMS) during their ICM externship. All the participants underwent two random draws (the 1st one for assessment tool to be started (PMP or CC) and the 2nd for the passage order for PMP. Two PMPs versus two grouped QCMs-CC were prepared and a satisfaction questionnaire was distributed. The main judgment criterion was the effect of each assessment tool on the students' decision-making process. This focused on the relevance of the elements provided by each technique, the implication and the difficulty felt. The secondary endpoint was the scores taken for each tool tested. RESULTS: 20 students were included. All participants had previous experience with PMPs and only nine were familiar with grouped MCQs-CC. PMP scores were 14.9 for the 1st theme and 15.8 for the 2nd theme. The median of the grouped MCQs-CC scores was 14 [12-16] for both. The scores didn't differ between the two techniques. For the 1st theme: the scores were negatively correlated (r=-0.58 and p=0.007). Students felt a better satisfaction for PMP evaluation (p<10-3), the elements provided by PMP were more relevant for decision-making process (p<10-3), the involvement was more felt with PMP (p<10-3) and difficulty was more felt with CCs (p<10-3). The effect of PMP was found to be significant on clinical reasoning (n=36), self-assessment (n=38), problem solving (n=40) and decision making (n=39). Students recommended strongly PMP as a tool of evaluation in ICM (p<10-3). CONCLUSION: scores were comparable between the two tested techniques. The positive perception of students regarding PMP encourages its generalization and teacher training must be strengthened.

Results of the Last 5 Years (2018-2022) of the Specialist Title Exam of The Brazilian College of Surgeons.

The article discusses the evolution of the Brazilian College of Surgeons (CBC) specialist title exam, highlighting the importance of evaluating not only theoretical knowledge, but also the practical skills and ethical behavior of candidates. The test was instituted in 1971, initially with only the written phase, and later included the oral practical test, starting with the 13th edition in 1988. In 2022, the assessment process was improved by including the use of simulated stations in the practical test, with the aim of assessing practical and communication skills, as well as clinical reasoning, in order to guarantee excellence in the assessment of surgeons training. The aim of this study is to demonstrate the performance of candidates in the last five years of the Specialist Title Test and to compare the performance results between the different surgical training groups of the candidates. The results obtained by candidates from the various categories enrolled in the test in the 2018 to 2022 editions were analyzed. There was a clear and statistically significant difference between doctors who had completed three years of residency recognized by the Ministry of Education in relation to the other categories of candidates for the Specialist Title..

Situational simulation teaching effectively improves dental students' non-operational clinical competency and objective structured clinical examination performance.

BACKGROUND: Appropriate communication with dental patients enhances treatment outcomes and patient satisfaction. Implementing simulated patient interviews courses can improve patient-centered care and reduce conflict during clerkship training. Therefore, this study explored the relationship among student participation in a situational simulation course (SSC), academic performance, clerkship performance, and objective structured clinical examination (OSCE) performance. METHODS: This study was conducted with a sample of fifth-year dental students undergoing clerkship training. After implementing a situational simulation course to investigate the relationship among participation in SSC, academic performance, clerkship performance, and OSCE performance, a path analysis model was developed and tested. RESULTS: Eighty-seven fifth-year dental students were eligible for the SSC, and most (n = 70, 80.46%) volunteered to participate. The path analysis model revealed that academic performance had a direct effect on OSCE performance (ß = 0.281, P = 0.003) and clerkship performance (ß = 0.441, P < 0.001). In addition, SSC teaching had a direct effect on OSCE performance (ß = 0.356, P < 0.001). CONCLUSIONS: SSCs can enhance dental students' non-operational clinical competency and OSCE performance effectively. Simulated patient encounters with feedback, incorporated into the dental curricula, have led to improved communication. Based on our findings, we suggest implementing SSC teaching before the OSCE to improve communication and cognitive skills.

Moving Beyond Hemangioma: Interactive, Multidisciplinary, Case-Based Teaching in Vascular Anomalies for Pediatric Residents.

Introduction: Vascular anomalies are a spectrum of disorders, including vascular tumors and malformations, that often require multispecialty care. The rarity and variety of these lesions make diagnosis, treatment, and management challenging. Despite the recognition of the medical complexity and morbidity associated with vascular anomalies, there is a general lack of education on the subject for pediatric primary care and subspecialty providers. A needs assessment and the lack of an available standardized teaching tool presented an opportunity to create an educational workshop for pediatric trainees using the POGIL (process-oriented guided inquiry learning) framework. Methods: We developed a 2-hour workshop consisting of an introductory didactic followed by small- and large-group collaboration and case-based discussion. The resource included customizable content for learning assessment and evaluation. Residents completed pre- and posttest assessments of content and provided written evaluations of the teaching session. Results: Thirty-four learners in pediatrics participated in the workshop. Session evaluations were positive, with Likert responses of 4.6-4.8 out of 5 on all items. Pre- and posttest comparisons of four content questions showed no overall statistically significant changes in correct response rates. Learners indicated plans to use the clinical content in their practice and particularly appreciated the interactive teaching forum and the comprehensive overview of vascular anomalies. Discussion: Vascular anomalies are complex, potentially morbid, and often lifelong conditions; multispecialty collaboration is key to providing comprehensive care for affected patients. This customizable resource offers a framework for trainees in pediatrics to appropriately recognize, evaluate, and refer patients with vascular anomalies.

Task Oriented Assessment of Clinical Skills (TOACS): A Modified Form of OSCE.

College of Physicians and Surgeons, Pakistan (CPSP) is a premier postgraduate medical institution of the country. It introduced Objective Structured Clinical Examination (OSCE) in the 1990s, and later came up with its modified form known as Task Oriented Assessment of Clinical Skills (TOACS). This modified assessment has been incorporated in clinical examinations of its majority fellowship programmes. Despite the use of TOACS for so many years at CPSP, it is surprising to note that this form of assessment does not appear in the literature. The objective of this viewpoint is to describe the rationale for the development of TOACS and to compare its structure and functions with OSCE. Key Words: Medical education, Assessment, Objective Structured Clinical Examination, Interactive, Task Oriented Assessment of Clinical Skills.

The Revival of Essay-Type Questions in Medical Education: Harnessing Artificial Intelligence and Machine Learning.

OBJECTIVE: To analyse and compare the assessment and grading of human-written and machine-written formative essays. STUDY DESIGN: Quasi-experimental, qualitative cross-sectional study. Place and Duration of the Study: Department of Science of Dental Materials, Hamdard College of Medicine & Dentistry, Hamdard University, Karachi, from February to April 2023. METHODOLOGY: Ten short formative essays of final-year dental students were manually assessed and graded. These essays were then graded using ChatGPT version 3.5. The chatbot responses and prompts were recorded and matched with manually graded essays. Qualitative analysis of the chatbot responses was then performed. RESULTS: Four different prompts were given to the artificial intelligence (AI) driven platform of ChatGPT to grade the summative essays. These were the chatbot's initial responses without grading, the chatbot's response to grading against criteria, the chatbot's response to criteria-wise grading, and the chatbot's response to questions for the difference in grading. Based on the results, four innovative ways of using AI and machine learning (ML) have been proposed for medical educators: Automated grading, content analysis, plagiarism detection, and formative assessment. ChatGPT provided a comprehensive report with feedback on writing skills, as opposed to manual grading of essays. CONCLUSION: The chatbot's responses were fascinating and thought-provoking. AI and ML technologies can potentially supplement human grading in the assessment of essays. Medical educators need to embrace AI and ML technology to enhance the standards and quality of medical education, particularly when assessing long and short essay-type questions. Further empirical research and evaluation are needed to confirm their effectiveness. KEY WORDS: Machine learning, Artificial intelligence, Essays, ChatGPT, Formative assessment.