Developing an electronic portfolio of learning for family medicine training in South Africa.

Workplace-based assessment has become increasingly crucial in the postgraduate training of specialists in South Africa, particularly for family physicians. The development of a Portfolio of Learning (PoL) has been a central focus within the discipline of family medicine for over a decade. Initially, a paper-based portfolio was adopted to collect evidence of learning for 50 out of 85 agreed exit-level outcomes. Stellenbosch University led the conversion of this portfolio into an electronic format, known as e-PoL, utilising Scorion software. The e-PoL was successfully implemented in the Western and Eastern Cape regions and was subsequently adopted nationally under the coordination of the South African Academy of Family Physicians. In 2023, the e-PoL underwent a redesign to gather evidence of learning for 22 entrustable professional activities (EPAs). Key insights from this development process underscore the importance of the PoL in supporting assessment-for-learning rather than merely assessment-of-learning. This necessitates features for feedback and interaction, ensuring that the PoL functions beyond a mere repository of forms. Additionally, the e-PoL should facilitate triangulation, aggregation, and saturation of data points to effectively measure EPAs. Furthermore, the PoL has not only documented learning but has also played a pivotal role in guiding the development of clinical training by explicitly outlining expectations for both registrars and supervisors. While the initial design and development costs are significant, operational costs become affordable when shared across all training programmes.

Use of Multiple-Choice Items in Summative Examinations: Questionnaire Survey Among German Undergraduate Dental Training Programs.

Background: Multiple-choice examinations are frequently used in German dental schools. However, details regarding the used item types and applied scoring methods are lacking. Objective: This study aims to gain insight into the current use of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programs. Methods: A paper-based 10-item questionnaire regarding the used assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the deans of studies and to the heads of the Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using the Fisher exact test (P<.05). Results: The response rate amounted to 90% (27/30 dental schools). All respondent dental schools used multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (24/27, 89%), which accounted for the largest number of items in approximately half of the dental schools (13/27, 48%). Further item types (eg, conventional multiple-select items, Multiple-True-False, and Pick-N) were only used by fewer dental schools (≤67%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit and requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (14/19, 74% vs 4/8, 50%). However, this difference was statistically not significant (P=.38). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (15/27, 56%). Not a single school used alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (15/27, 56%). Conclusions: Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.

Implementation of the São Paulo Nursing Courses Consortium for the Progress Test: experience report.

OBJECTIVE: To report the experience of implementing the São Paulo Nursing Courses Consortium for the Progress Test. METHOD: This is an experience report of the consortium's work in Progress Test preparation and application for Public Schools of Nursing in São Paulo in 2019, 2021 and 2022, with a descriptive analysis of the work process and the results obtained. RESULTS: The consortium's activities are structured into the following stages: planning; theme review; distributing and requesting questions; professor training; question elaboration; question reception; question selection; question validation; student registration; test application; analysis and dissemination of results. A total of 57.3% of enrolled students participated. There was a predominance of questions of medium difficulty and a gradual progression in the level of discrimination of the questions, with, in 2022, 82.5% being considered adequate. FINAL CONSIDERATIONS: The consortium has allowed the test to be applied interinstitutionally, with greater scope, accuracy, and quality of questions. Through this experience, it is expected to encourage progress testing in undergraduate nursing courses in other contexts.

A Mini Chalk Talk Workshop for Fourth-Year Medical Students: Facilitating the Transition From Student to Resident Educator.

Introduction: There is increasing recognition that incoming interns benefit from formal training in teaching skills during UME. Many medical schools have capstone courses well suited for teacher-training content. Mini chalk talks (MCTs) are a common clinical teaching modality requiring a variety of teaching skills. We developed a session for our institution's capstone course in which students prepared and delivered MCTs. Methods: The voluntary flipped classroom session was offered virtually in 2021 and in person in 2022. Before the session, students reviewed materials on creating effective MCTs and developed and practiced their own MCT. During the 90-minute session, students presented their MCT to a group of students in the same or similar future specialties and received feedback from their peers and a facilitator. Results: Twenty-six percent of graduating students (95 of 370) in 16 specialties participated. Students had a statistically significant increase in confidence delivering effective MCTs (p < .01). On a 5-point Likert scale (1 = did not learn, 5 = a great amount), students' mean ratings of clinical knowledge and teaching skills gained from the session were 4.4 and 4.5, respectively. Qualitative feedback highlighted the benefits of receiving feedback on teaching (31 of 77 respondents, 40%), practicing teaching skills (21 of 77, 27%), and experiencing other students' MCTs (13 of 77, 17%). Discussion: Our MCT session provides a versatile, resource-efficient method of supporting students in transitioning to the role of resident educators. It also offers them an opportunity to receive valuable feedback on their teaching in a low-stakes environment.

Creation of a Common Framework to Evaluate Initial Registered Nurse Role Competency.

Assessment of initial nursing competency is essential to safe nursing practice yet often focuses on psychomotor skill acquisition. A multistate health system created a competency strategy based on a comprehensive conceptualization of competency using the American Nursing Association scope and standards of nursing practice. This approach allows for the broad application of a standard competency assessment tool across diverse nursing specialties and provides a framework for nursing professional development practitioners to implement in their organizations.

Variance due to the examination conditions and factors associated with success in objective structured clinical examinations (OSCEs): first experiences at Paris-Saclay medical school.

BACKGROUND: We aimed to measure the variance due to examination conditions during the first sessions of objective structured clinical examinations (OSCEs) performed at a French medical school and identify factors associated with student success. METHODS: We conducted a retrospective, observational study using data from the first three OSCEs sessions performed at Paris-Saclay medical school in 2021 and 2022. For all sessions (each organized in 5 parallel circuits), we tested a circuit effect using a linear mixed-effects model adjusted for sex and the average academic level of students (according to written tests). Then, we studied the factors associated with student success at one station using a multivariate linear mixed-effects model, including the characteristics of students, assessors, and standardized patients. RESULTS: The study included three OSCEs sessions, with 122, 175, and 197 students and a mean (± SD) session score of 13.7(± 1.5)/20, 12.7(± 1.7)/20 and 12.7(± 1.9)/20, respectively. The percentage of variance due to the circuit was 6.5%, 18.2% (statistically significant), and 3.8%, respectively. For all sessions, the student's average level and station scenario were significantly associated with the score obtained in a station. Still, specific characteristics of assessors or standardized patients were only associated with the student's score in April 2021 (first session). CONCLUSION: The percentage of the variance of students' performance due to the examination conditions was significant in one out of three of the first OSCE sessions performed at Paris-Saclay medical school. This result seems more related to individual behaviors rather than specific characteristics of assessors or standardized patients, highlighting the need to continue training teaching teams. NATIONAL CLINICAL TRIAL NUMBER: Not applicable.

A real-world test of artificial intelligence infiltration of a university examinations system: A "Turing Test" case study.

The recent rise in artificial intelligence systems, such as ChatGPT, poses a fundamental problem for the educational sector. In universities and schools, many forms of assessment, such as coursework, are completed without invigilation. Therefore, students could hand in work as their own which is in fact completed by AI. Since the COVID pandemic, the sector has additionally accelerated its reliance on unsupervised 'take home exams'. If students cheat using AI and this is undetected, the integrity of the way in which students are assessed is threatened. We report a rigorous, blind study in which we injected 100% AI written submissions into the examinations system in five undergraduate modules, across all years of study, for a BSc degree in Psychology at a reputable UK university. We found that 94% of our AI submissions were undetected. The grades awarded to our AI submissions were on average half a grade boundary higher than that achieved by real students. Across modules there was an 83.4% chance that the AI submissions on a module would outperform a random selection of the same number of real student submissions.

Comparative accuracy of ChatGPT-4, Microsoft Copilot and Google Gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study.

BACKGROUND: Artificial intelligence (AI) chatbots are emerging educational tools for students in healthcare science. However, assessing their accuracy is essential prior to adoption in educational settings. This study aimed to assess the accuracy of predicting the correct answers from three AI chatbots (ChatGPT-4, Microsoft Copilot and Google Gemini) in the Italian entrance standardized examination test of healthcare science degrees (CINECA test). Secondarily, we assessed the narrative coherence of the AI chatbots' responses (i.e., text output) based on three qualitative metrics: the logical rationale behind the chosen answer, the presence of information internal to the question, and presence of information external to the question. METHODS: An observational cross-sectional design was performed in September of 2023. Accuracy of the three chatbots was evaluated for the CINECA test, where questions were formatted using a multiple-choice structure with a single best answer. The outcome is binary (correct or incorrect). Chi-squared test and a post hoc analysis with Bonferroni correction assessed differences among chatbots performance in accuracy. A p-value of < 0.05 was considered statistically significant. A sensitivity analysis was performed, excluding answers that were not applicable (e.g., images). Narrative coherence was analyzed by absolute and relative frequencies of correct answers and errors. RESULTS: Overall, of the 820 CINECA multiple-choice questions inputted into all chatbots, 20 questions were not imported in ChatGPT-4 (n = 808) and Google Gemini (n = 808) due to technical limitations. We found statistically significant differences in the ChatGPT-4 vs Google Gemini and Microsoft Copilot vs Google Gemini comparisons (p-value < 0.001). The narrative coherence of AI chatbots revealed "Logical reasoning" as the prevalent correct answer (n = 622, 81.5%) and "Logical error" as the prevalent incorrect answer (n = 40, 88.9%). CONCLUSIONS: Our main findings reveal that: (A) AI chatbots performed well; (B) ChatGPT-4 and Microsoft Copilot performed better than Google Gemini; and (C) their narrative coherence is primarily logical. Although AI chatbots showed promising accuracy in predicting the correct answer in the Italian entrance university standardized examination test, we encourage candidates to cautiously incorporate this new technology to supplement their learning rather than a primary resource. TRIAL REGISTRATION: Not required.

Embedding Assessment Literacy Can Enhance Graduate Attribute Development in a Biomedical Sciences Curriculum.

This paper describes the successful implementation of an assessment literacy strategy within a Biomedical Sciences degree. Teaching was aligned with an assessment literacy framework and aimed to prepare undergraduates for a literature comprehension assessment. Students were introduced to the assessment purpose and an adapted Miller's pyramid model illustrated how the assessment contributed to competency development during their degree. Students read primary research papers and answered questions relating to the publications. They were then introduced to the processes of assessment and collaboratively graded answers of different standards. Finally, student and faculty grades were compared, differences considered, and key characteristics of answers discussed. Most students reported that they understood more about assessment standards than prior to the intervention [139/159 (87.4%)] and felt it had helped prepare them for their exam [138/159 (86.8%)]. The majority also reported they had increased confidence in evaluating data [118/159 (74%)], communicating their reasoning [113/159 (71%)] and considering what a reader needs to know [127/159 (79.9%)]. Students were asked to state the most important thing they had learned from the assessment literacy teaching. Notably, no responses referred to domain-specific knowledge. 129 free text responses were mapped to the University of Edinburgh graduate attribute framework. 93 (72%) statements mapped to the graduate attribute category "Research and Enquiry," 66 (51.16%) mapped to "Communication" and 21 (16.27%) mapped to "Personal and Intellectual Autonomy." To explore any longer-term impact of the assessment literacy teaching, a focus group was held with students from the same cohort, 2 years after the original intervention. Themes from this part of the study included that teaching had provided insights into standards and expectations for the assessment and the benefits of domain specific knowledge. A variety of aspects related to graduate attributes were also identified. Here, assessment literacy as a vehicle for graduate attribute development was an unexpected outcome. We propose that by explicitly engaging students with purpose, process, standards, and expectations, assessment literacy strategies may be used to successfully raise awareness of developmental progression, and enhance skills, aptitudes, and dispositions beneficial to Biomedical Sciences academic achievement and life after university.

Accuracy of progress monitoring decision rules to evaluate response to instruction with two computer adaptive tests.

Computer adaptive tests have become popular assessments to screen students for academic risk. Research is emerging regarding their use as progress monitoring tools to measure response to instruction. We evaluated the accuracy of the trend-line decision rule when applied to outcomes from a frequently used reading computer adaptive test (i.e., Star Reading [SR]) and frequently used math computer adaptive test (i.e., Star Math [SM]). Analyses of extant SR and SM data were conducted to inform conditions for simulations to determine the number of assessments required to yield sufficient sensitivity (i.e., probability of recommending an instructional change when a change was warranted) and specificity (i.e., probability of recommending maintaining an intervention when a change was not warranted) when comparing performance to goal lines based upon a future target score (i.e., benchmark) as well as normative comparisons (50th and 75th percentiles). The extant dataset of SR outcomes consisted of monthly progress monitoring data from 993 Grade 3, 804 Grade 4, and 709 Grade 5 students from multiple states in the United States northwest. Data for SM were also drawn from the northwest and contained outcomes from 518 Grade 3, 474 Grade 4, and 391 Grade 5 students. Grade level samples were predominately White (range = 59.89%-67.72%) followed by Latinx (range = 9.65%-15.94%). Results of simulations suggest that when data were collected once a month, seven, eight, and nine observations were required to support low-stakes decisions with SR for Grades 3, 4, and 5, respectively. For SM, nine, ten, and eight observations were required for Grades, 3, 4, and 5, respectively. Given the length of time required to support reasonably accurate decisions, recommendations to consider other types of assessments and decision-making frameworks for academic progress monitoring are provided.

[The involvement of patients in the assessment of nursing students during their practical training: a scoping review]. / Il coinvolgimento dei pazienti nel processo di valutazione dello studente di Infermieristica in tirocinio: una scoping review.

. The involvement of patients in the assessment of nursing students during their practical training: a scoping review. INTRODUCTION: In health care education, several experiences of patient involvement in students' assessment are documented internationally: this represents a challenge for nursing education. OBJECTIVE: To analyze user involvement experiences in nursing student assessment in order to identify: a) potential benefits and challenges, b) assessment methods and tools, c) assessed skills. METHODS: A scoping review was conducted according to the methodology proposed by the Joanna Briggs Institute. Methods and findings have been reported according to the PRISMA-ScR (Systematic reviews and Meta-Analyses extension for Scoping Reviews) statement. RESULTS: The results of the 22 included studies, published between 1995 and 2021, mainly in the United Kingdom and Europe, indicate that user' engagement is an experience that can be useful and meaningful. However, it is not without challenges such as the difficulty of providing critical feedback, student anxiety, and the investment of time and resources required. Assessments can be collected through scales, questionnaires, or interviews, focusing on interpersonal skills such as listening and respect, allowing tutors to gather surprising details about students' performance. CONCLUSIONS: Patients can provide original insights into nursing student's caring behaviors and can be a source of learning based on valuing the point of view of care receivers. Implementing patient participation in this process requires clarity, commited adherence and rigorous planning.

The Effect of Critical Task Auto-failure Criteria on Medical Evaluation Methods in the Pararescue Schoolhouse.

BACKGROUND: Medical training and evaluation are important for mission readiness in the pararescue career field. Because evaluation methods are not standardized, evaluation methods must align with training objectives. We propose an alternative evaluation method and discuss relevant factors when designing military medical evaluation metrics. METHODS: We compared two evaluation methods, the traditional checklist (TC) method used in the pararescue apprentice course and an alternative weighted checklist (AWC) method like that used at the U.S. Army static line jumpmaster course. The AWC allows up to two minor errors, while critical task errors result in autofailure. We recorded 168 medical scenarios during two Apprentice course classes and retroactively compared the two evaluation methods. RESULTS: Despite the possibility of auto-failure with the AWC, there was no significant difference between the two evaluation methods, and both showed similar overall pass rates (TC=50% pass, AWC=48.8% pass, p=.41). The two evaluation methods yielded the same result for 147 out of 168 scenarios (87.5%). CONCLUSIONS: The AWC method strongly emphasizes critical tasks without significantly increasing failures. It may provide additional benefits by being more closely aligned with our training objectives while providing quantifiable data for a longitudinal review of student performance.

OWIED System to Facilitate the Entrustment Decision in EPA Implementation.

BACKGROUND: Entrustable professional activities (EPAs) are gaining attention as an approach for adequate implementation of a competency-based evaluation framework in nursing education. This study sought to develop an Online Web-assessment Interface for Entrustment Decision (OWIED) to facilitate the entrustment decision in EPA implementation. METHOD: A participatory qualitative action research design consisting of two phases was used for this study. The exploration phase was conducted in close collaboration with stakeholders. Following the exploration phase, the primary researcher and a team of subject-matter experts in academic and information technology developed the OWIED system according to the exploration phase results. RESULTS: The necessary features that met the expectations of the stakeholders were identified and assisted in developing the OWIED system. CONCLUSION: OWIED may provide a valid tool to track and validate nursing student acquisition of core competencies and assist in making entrustment decisions as students fulfill their training requirements in their academic program. [J Nurs Educ. 2024;63(6):356-365.].

Test Analysis Through the Lens of Objective Compassion.

ABSTRACT: Test construction and test reviews are responsibilities nursing faculty arduously undertake, with an obligation to give appropriate effort and time to prepare and review exams. During test review, item analysis and statistical analysis offer valuable empirical information about the exam. However, objective compassion is also needed and can be demonstrated through careful test question construction and item analysis. Furthermore, compassion is needed in preparing students for the Next Generation NCLEX-RN (NGN) and constructing exams that appropriately test students' clinical judgment.

Development of examination objectives for the Korean paramedic and emergency medical technician examination: a survey study.

PURPOSE: The duties of paramedics and emergency medical technicians (P&EMTs) are continuously changing due to developments in medical systems. This study presents evaluation goals for P&EMTs by analyzing their work, especially the tasks that new P&EMTs (with less than 3 years' experience) find difficult, to foster the training of P&EMTs who could adapt to emergency situations after graduation. METHODS: A questionnaire was created based on prior job analyses of P&EMTs. The survey questions were reviewed through focus group interviews, from which 253 task elements were derived. A survey was conducted from July 10, 2023 to October 13, 2023 on the frequency, importance, and difficulty of the 6 occupations in which P&EMTs were employed. RESULTS: The P&EMTs' most common tasks involved obtaining patients' medical histories and measuring vital signs, whereas the most important task was cardiopulmonary resuscitation (CPR). The task elements that the P&EMTs found most difficult were newborn delivery and infant CPR. New paramedics reported that treating patients with fractures, poisoning, and childhood fever was difficult, while new EMTs reported that they had difficulty keeping diaries, managing ambulances, and controlling infection. CONCLUSION: Communication was the most important item for P&EMTs, whereas CPR was the most important skill. It is important for P&EMTs to have knowledge of all tasks; however, they also need to master frequently performed tasks and those that pose difficulties in the field. By deriving goals for evaluating P&EMTs, changes could be made to their education, thereby making it possible to train more capable P&EMTs.

Evaluation of the Oral Health Education Programme for Nurses using an Oral Lesion Simulator.

PURPOSE: This study aimed to investigate the usefulness of a newly developed oral simulator for nursing students' oral assessment education on oral diseases and symptoms. MATERIALS AND METHODS: The participants were first-year students (n=105) at a nursing school in Japan. Ten identical oral simulators with angular cheilitis, missing teeth, dental caries, calculus, periodontitis, hypoglossal induration, food debris, and crust formation were created by a team of dentists. After a 45-minute lecture programme for oral assessment performance with the Oral Health Assessment Tool (OHAT), the ability test with the simulators and the OHAT as well as test feedback were conducted in a 30-minute practical programme. To evaluate the effectiveness of the programmes, questionnaires and ability tests with slides of oral images were conducted at baseline and after the programme. RESULTS: Ninety-nine students (94.3%) participated in this study. The results of the ability test with the simulators and the OHAT in the practical programme showed that the correct answer rates of assessing tongue, gingiva, present teeth, and oral pain were less than 40%. Their levels of confidence, perception, and oral assessment performance were statistically significantly higher after the programmes than they were at baseline. Their level of confidence in assessing the need for dental referral had the largest increase in scores compared to the lowest scores at baseline in the nine post-programme assessment categories. CONCLUSIONS: This study identified several problems with nursing students' oral assessment skills and improvements of their oral assessment confidence, perceptions and performance.

In-depth analysis of ChatGPT's performance based on specific signaling words and phrases in the question stem of 2377 USMLE step 1 style questions.

ChatGPT has garnered attention as a multifaceted AI chatbot with potential applications in medicine. Despite intriguing preliminary findings in areas such as clinical management and patient education, there remains a substantial knowledge gap in comprehensively understanding the chances and limitations of ChatGPT's capabilities, especially in medical test-taking and education. A total of n = 2,729 USMLE Step 1 practice questions were extracted from the Amboss question bank. After excluding 352 image-based questions, a total of 2,377 text-based questions were further categorized and entered manually into ChatGPT, and its responses were recorded. ChatGPT's overall performance was analyzed based on question difficulty, category, and content with regards to specific signal words and phrases. ChatGPT achieved an overall accuracy rate of 55.8% in a total number of n = 2,377 USMLE Step 1 preparation questions obtained from the Amboss online question bank. It demonstrated a significant inverse correlation between question difficulty and performance with rs = -0.306; p < 0.001, maintaining comparable accuracy to the human user peer group across different levels of question difficulty. Notably, ChatGPT outperformed in serology-related questions (61.1% vs. 53.8%; p = 0.005) but struggled with ECG-related content (42.9% vs. 55.6%; p = 0.021). ChatGPT achieved statistically significant worse performances in pathophysiology-related question stems. (Signal phrase = "what is the most likely/probable cause"). ChatGPT performed consistent across various question categories and difficulty levels. These findings emphasize the need for further investigations to explore the potential and limitations of ChatGPT in medical examination and education.

Impact of hospital internships on success in university summative objective structured clinical examinations: Large-scale experience in a French medical school.

BACKGROUND: Objective structured clinical examinations (OSCEs) are extensively used in many medical schools worldwide with the stated objective to assess students' clinical skills acquired during internships. The objective of the present study was to assess the factors associated with success in university summative OSCEs, especially the impact of previous hospital internships in corresponding disciplines and supervision during internships. METHODS: This was a cross-sectional study assessing the results in the summative OSCEs of 4th year medical students during the 2021-2022 academic year in a French medical school. The summative OSCEs included five stations for each student. Each student answered a survey at the end of summative OSCEs about previous internships, the supervision they had and perceived difficulty levels for each station. The scores in each station were assessed according to previous hospital internships in the corresponding discipline. Analysis of predictive factors of success in OSCEs, defined by a score ≥ 10/20 at each station, were performed using univariate and multivariate logistic regression. RESULTS: Out of the 220 students participating in the summative OSCEs, 182 (83%) answered the survey. One hundred and forty-four (79%) of these students had carried out hospital internships in at least one of the disciplines evaluated during the OSCEs. Students having completed an internship in the corresponding discipline had significantly higher OSCEs scores for interrogation, communication, therapeutic education and procedure stations compared to those who had not. Previous internship in corresponding disciplines was independently associated with success in OSCEs in interrogation (OR 9.45 [1.34-66.8] p = 0.02), clinical examination (OR 6.93 [1.88-25.57] p = 0.004, and therapeutic education (OR 3.09 [1.22-7.82] p = 0.02) stations. CONCLUSION: Previous hospital internships in the discipline evaluated by the OSCEs are associated with success in summative OSCEs. This reinforces the importance of student involvement during their hospital internships.

Examiner workload comparison: three structured oral examination formats for the European diploma in anaesthesiology and intensive care.

The COVID-19 pandemic triggered transformations in academic medicine, rapidly adopting remote teaching and online assessments. Whilst virtual environments show promise in evaluating medical knowledge, their impact on examiner workload is unclear. This study explores examiner's workload during different European Diploma in Anaesthesiology and Intensive Care Part 2 Structured Oral Examinations formats. We hypothesise that online exams result in lower examiner's workload than traditional face-to-face methods. We also investigate workload structure and its correlation with examiner characteristics and marking performance. In 2023, examiner's workload for three examination formats (face-to-face, hybrid, online) using the NASA TLX instrument was prospectively evaluated. The impact of examiner demographics, candidate scoring agreement, and examination scores on workload was analysed. The overall NASA TLX score from 215 workload measurements in 142 examiners was high at 59.61 ± 14.13. The online examination had a statistically higher workload (61.65 ± 12.84) than hybrid but not face-to-face. Primary contributors to workload were mental and temporal demands, and effort. Online exams were associated with elevated frustration. Male examiners and those spending more time on exam preparation experienced a higher workload. Multiple diploma specialties and familiarity with European Diploma in Anaesthesiology and Intensive Care exams were protective against high workload. Perceived workload did not impact marking agreement or examination scores across all formats. Examiners experience high workload. Online exams are not systematically associated with decreased workload, likely due to frustration. Despite workload differences, no impact on examiner's performance or examination scores was found. The hybrid examination mode, combining face-to-face and online, was associated with a minor but statistically significant workload reduction. This hybrid approach may offer a more balanced and efficient examination process while maintaining integrity, cost savings, and increased accessibility for candidates.