Your browser doesn't support javascript.
loading
Show: 20 | 50 | 100
Results 1 - 20 de 5.686
Filter
1.
Syst Rev ; 13(1): 131, 2024 May 14.
Article in English | MEDLINE | ID: mdl-38745201

ABSTRACT

BACKGROUND: The current paradigm of competency-based medical education and learner-centredness requires learners to take an active role in their training. However, deliberate and planned continual assessment and performance improvement is hindered by the fragmented nature of many medical training programs. Attempts to bridge this continuity gap between supervision and feedback through learner handover have been controversial. Learning plans are an alternate educational tool that helps trainees identify their learning needs and facilitate longitudinal assessment by providing supervisors with a roadmap of their goals. Informed by self-regulated learning theory, learning plans may be the answer to track trainees' progress along their learning trajectory. The purpose of this study is to summarise the literature regarding learning plan use specifically in undergraduate medical education and explore the student's role in all stages of learning plan development and implementation. METHODS: Following Arksey and O'Malley's framework, a scoping review will be conducted to explore the use of learning plans in undergraduate medical education. Literature searches will be conducted using multiple databases by a librarian with expertise in scoping reviews. Through an iterative process, inclusion and exclusion criteria will be developed and a data extraction form refined. Data will be analysed using quantitative and qualitative content analyses. DISCUSSION: By summarising the literature on learning plan use in undergraduate medical education, this study aims to better understand how to support self-regulated learning in undergraduate medical education. The results from this project will inform future scholarly work in competency-based medical education at the undergraduate level and have implications for improving feedback and supporting learners at all levels of competence. SCOPING REVIEW REGISTRATION: Open Science Framework osf.io/wvzbx.


Subject(s)
Education, Medical, Undergraduate , Learning , Education, Medical, Undergraduate/methods , Humans , Clinical Competence , Competency-Based Education/methods
2.
J Nurs Educ ; 63(5): 320-327, 2024 May.
Article in English | MEDLINE | ID: mdl-38729143

ABSTRACT

BACKGROUND: Accuracy is needed with medication administration, a skill that involves rule-based habits and clinical reasoning. This pilot study investigated the use of an evidence-based checklist for accuracy with oral medication administration and error reporting among prelicensure nursing students. Checklist items were anchored in the mnemonic C-MATCH-REASON© (Client, Medication, ADRs, Time, Client History, Route, Expiration date, Amount, Site, Outcomes, Notation). METHOD: Nineteen participants randomly assigned to crossover sequence AB or BA (A: checklist; B: no checklist) practiced simulation scenarios with embedded errors. Nursing faculty used an observation form to track error data. RESULTS: Using the C-MATCH-REASON© checklist compared with not using the checklist supported rule adherence (p = .005), knowledge-based error reduction (p = .011), and total error reduction (p = .010). The null hypothesis was not rejected for errors found (p = .061) nor reported (p = .144), possibly due to sample size. CONCLUSION: C-MATCH-REASON© was effective for error reduction. Study replication with a larger sample is warranted. [J Nurs Educ. 2024;63(5):320-327.].


Subject(s)
Checklist , Clinical Competence , Medication Errors , Humans , Pilot Projects , Clinical Competence/standards , Medication Errors/prevention & control , Education, Nursing, Baccalaureate , Students, Nursing/statistics & numerical data , Female , Male , Nursing Education Research , Cross-Over Studies , Competency-Based Education
4.
J Dent Educ ; 88(5): 639-653, 2024 May.
Article in English | MEDLINE | ID: mdl-38693898

ABSTRACT

PURPOSE: Entrustable professional activities (EPAs) are discrete clinical tasks that can be evaluated to help define readiness for independent practice in the health professions and are intended to increase trust in the dental graduate. EPAs provide a framework that bridges competencies to clinical practice. This report describes the work of the American Dental Education Association (ADEA) Compendium EPA Workgroup to develop a list of EPAs for dental education and supportive resources, including specifications and a glossary. METHODS: Preliminary work including literature and resource review, mapping of existing competencies, and review of other health professions' EPAs informed the development of our EPAs list. Workgroup members achieved consensus using a modified Delphi process. A Qualtrics survey using a validated rubric for the assessment of EPAs as described in peer-reviewed literature was used. Dental educators, including academic deans, were surveyed for feedback on the content and format of the EPAs. RESULTS: Based on findings in the literature analysis of existing EPAs and competencies in health professions, a list of EPAs was developed along with a description of specifications. The EPA workgroup (nine members from multiple institutions) used the Delphi process in receiving feedback from various experts. A list of 11 core EPAs was vetted by dental educators including academic deans (n = âˆ¼23), and the process of development was reviewed by EPAs experts outside dental education. A glossary was developed to align language. CONCLUSION: These EPAs define the scope of dental practice. This report represents Phase 1 of the EPA framework development and vetting process. Future directions will include a broader vetting of the EPA list, faculty development, and national standardized technology that support this work to optimize implementation.


Subject(s)
Clinical Competence , Competency-Based Education , Education, Dental , United States , Education, Dental/standards , Clinical Competence/standards , Competency-Based Education/standards , Humans , Delphi Technique , Societies, Dental
5.
J Surg Educ ; 81(6): 841-849, 2024 Jun.
Article in English | MEDLINE | ID: mdl-38664173

ABSTRACT

OBJECTIVE: This study aimed to identify what best practices facilitate implementation of Entrustable Professional Activities (EPAs) into surgical training programs. DESIGN: This is a mixed methods study utilizing both survey data as well as semi-structured interviews of faculty and residents involved in the American Board of Surgery (ABS) EPA pilot study. SETTING: From 2018 to 2020, the ABS conducted a pilot that introduced five EPAs across 28 general surgery training programs. PARTICIPANTS: All faculty members and residents at the 28 pilot programs were invited to participate in the study. RESULTS: About 117 faculty members and 79 residents responded to the survey. The majority of faculty (81%) and residents (66%) felt that EPAs were useful and were a valuable addition to training. While neither group felt that EPAs were overly time consuming to complete, residents did report difficulty incorporating them into their daily workflow (44%). Semi-structured interviews found that programs that focused on faculty and resident -development and utilized frequent reminders about the importance and necessity of EPAs tended to perform better. CONCLUSIONS: EPA implementation is feasible in general surgery training programs but requires significant effort and engagement from all levels of program personnel. As EPAs are implemented by the ABS nationally a focus on resident and faculty development will be critical to success.


Subject(s)
Faculty, Medical , General Surgery , Internship and Residency , General Surgery/education , Humans , Pilot Projects , Competency-Based Education , Male , Female , Clinical Competence , Attitude of Health Personnel , Education, Medical, Graduate/methods , Surveys and Questionnaires , United States
6.
J Surg Educ ; 81(6): 823-840, 2024 Jun.
Article in English | MEDLINE | ID: mdl-38679495

ABSTRACT

OBJECTIVE: Entrustable professional activities (EPAs) are a crucial component of contemporary postgraduate medical education with many surgery residency programs having implemented EPAs as a competency assessment framework to assess and provide feedback on the performance of their residents. Despite broad implementation of EPAs, there is a paucity of evidence regarding the impact of EPAs on the learners and learning environments. A first step in improving understanding of the use and impact of EPAs is by mapping the rising number of EPA-related publications from the field of surgery. The primary objective of this scoping review is to examine the nature, extent, and range of articles on the development, implementation, and assessment of EPAs. The second objective is to identify the experiences and factors that influence EPA implementation and use in practice in surgical specialties. DESIGN: Scoping review. Four electronic databases (Medline, Embase, Education Source, and ERIC) were searched on January 20, 2022, and then again on July 19, 2023. A quasi-statistical content analysis was employed to quantify and draw meaning from the information related to the development, implementation, assessment, validity, reliability, and experiences with EPAs in the workplace. PARTICIPANTS: A total of 42 empirical and nonempirical articles were included. RESULTS: Four thematic categories describe the topic areas in included articles related to: 1) the development and refinement of EPAs, including the multiple steps taken to develop and refine unique EPAs for surgery residency programs; 2) the methods for implementing EPAs; 3) outcomes of EPA use in practice; 4) barriers, facilitators, and areas for improvement for the implementation and use of EPAs in surgical education. CONCLUSIONS: This scoping review highlights the key trends and gaps from the rapidly increasing number of publications on EPAs in surgery residency, from development to their use in the workplace. Existing EPA studies lack a theoretical and/or conceptual basis; future development and implementation studies should adopt implementation science frameworks to better structure and operationalize EPAs within surgery residency programs.


Subject(s)
Clinical Competence , Competency-Based Education , Internship and Residency , Competency-Based Education/methods , General Surgery/education , Humans , Education, Medical, Graduate/methods
7.
BMC Med Educ ; 24(1): 427, 2024 Apr 22.
Article in English | MEDLINE | ID: mdl-38649850

ABSTRACT

BACKGROUND: Work-integrated learning (WIL) is widely accepted and necessary to attain the essential competencies healthcare students need at their future workplaces. Yet, competency-based education (CBE) remains complex. There often is a focus on daily practice during WIL. Hereby, continuous competency development is at stake. Moreover, the fact that competencies need to continuously develop is often neglected. OBJECTIVES: To ultimately contribute to the optimization of CBE in healthcare education, this study aimed at examining how competency development during WIL in healthcare education could be optimized, before and after graduation. METHODS: Fourteen semi-structured interviews with 16 experts in competency development and WIL were carried out. Eight healthcare disciplines were included namely associate degree nursing, audiology, family medicine, nursing (bachelor), occupational therapy, podiatry, pediatrics, and speech therapy. Moreover, two independent experts outside the healthcare domain were included to broaden the perspectives on competency development. A qualitative research approach was used based on an inductive thematic analysis using Nvivo12© where 'in vivo' codes were clustered as sub-themes and themes. RESULTS: The analysis revealed eight types of requirements for effective and continuous competency development, namely requirements in the context of (1) competency frameworks, (2) reflection and feedback, (3) assessment, (4) the continuity of competency development, (5) mentor involvement, (6) ePortfolios, (7) competency development visualizations, and (8) competency development after graduation. It was noteworthy that certain requirements were fulfilled in one educational program whereas they were absent in another. This emphasizes the large differences in how competence-based education is taking shape in different educational programs and internship contexts. Nevertheless, all educational programs seemed to recognize the importance of ongoing competency development. CONCLUSION: The results of this study indicate that identifying and meeting the requirements for effective and continuous competency development is essential to optimize competency development during practice in healthcare education.


Subject(s)
Clinical Competence , Competency-Based Education , Humans , Clinical Competence/standards , Qualitative Research , Female , Male , Interviews as Topic , Curriculum
8.
BMC Med Educ ; 24(1): 436, 2024 Apr 22.
Article in English | MEDLINE | ID: mdl-38649947

ABSTRACT

BACKGROUND: The integration of Objective Structured Clinical Examinations (OSCEs) within the professional pharmacy program, contributes to assessing the readiness of pharmacy students for Advanced Pharmacy Practice Experiences (APPEs) and real-world practice. METHODS: In a study conducted at an Accreditation Council for Pharmacy Education (ACPE)-accredited Doctor of Pharmacy professional degree program, 69 students in their second professional year (P2) were engaged in OSCEs. These comprised 3 stations: best possible medication history, patient education, and healthcare provider communication. These stations were aligned with Entrustable Professional Activities (EPAs) and Ability Statements (AS). The assessment aimed to evaluate pharmacy students' competencies in key areas such as ethical and legal behaviors, general communication skills, and interprofessional collaboration. RESULTS: The formulation of the OSCE stations highlighted the importance of aligning the learning objectives of the different stations with EPAs and AS. The evaluation of students' ethical and legal behaviors, the interprofessional general communication, and collaboration showed average scores of 82.6%, 88.3%, 89.3%, respectively. Student performance on communication-related statements exceeded 80% in all 3 stations. A significant difference (p < 0.0001) was found between the scores of the observer and the SP evaluator in stations 1 and 2 while comparable results (p = 0.426) were shown between the observer and the HCP evaluator in station 3. Additionally, a discrepancy among the observers' assessments was detected across the 3 stations. The study shed light on challenges encountered during OSCEs implementation, including faculty involvement, resource constraints, and the necessity for consistent evaluation criteria. CONCLUSIONS: This study highlights the importance of refining OSCEs to align with EPAs and AS, ensuring a reliable assessment of pharmacy students' clinical competencies and their preparedness for professional practice. It emphasizes the ongoing efforts needed to enhance the structure, content, and delivery of OSCEs in pharmacy education. The findings serve as a catalyst for addressing identified challenges and advancing the effectiveness of OSCEs in accurately evaluating students' clinical readiness.


Subject(s)
Clinical Competence , Curriculum , Education, Pharmacy , Educational Measurement , Students, Pharmacy , Humans , Clinical Competence/standards , Educational Measurement/methods , Competency-Based Education
9.
Rev. Fund. Educ. Méd. (Ed. impr.) ; 27(2): 89-99, Abr. 2024. tab, graf
Article in Spanish | IBECS | ID: ibc-VR-26

ABSTRACT

Introducción: Pocas veces se indaga sobre el significado que tiene para los profesores recibir informes sobre sus competencias docentes en opinión de los médicos residentes. El objetivo de este estudio es comprender la manera en que losprofesores valoran y usan los datos de la evaluación docente (ED) para transformar sus prácticas educativas. Sujetos y métodos: Es un estudio cuantitativo descriptivo por medio de encuesta aplicada en línea durante septiembre y octubre de 2023. La población fue de 896 profesores titulares y adjuntos con nombramiento de 79 especialidades evaluados al menos por los residentes en una ocasión. Respondieron 491 profesores (54,8%). La estructura del instrumento tenía tres secciones: datos sociodemográficos y académicos, preguntas cerradas con escalas de tipo Likert categorizadas en cuatro temas y preguntas abiertas para comentarios escritos.Resultados: Para más del 85%, la ED es un referente que influye en su desempeño docente. El 93,3% está de acuerdo en que la ED ayuda a identificar fortalezas y debilidades. El 75% o más cree que las preguntas del cuestionario que responden los residentes se relacionan con sus prácticas educativas. Conclusiones: La ED es aceptada y valorada positivamente por los docentes, sus respuestas comparten el imaginario colectivo característico de los contextos de educación superior que consideran esta evaluación una estrategia para realimentar y mejorar su desempeño docente, así como para detectar fortalezas y áreas de oportunidad. Se requieren discusiones guiadas sobre las competencias docentes que vayan más allá de los informes individuales con el fin de visibilizar lasdinámicas académicas en la sede.(AU)


Introduction: The meaning of the data given by professors who receive teaching competencies reports in the opinion of resident physicians is rarely investigated. The aim of this study is to understand how teachers value and use teaching evaluation (TE) data to transform their educational practices.Subjects and methods: Quantitative descriptive study by means of an online survey applied during September and October 2023. The population consisted of 896 tenured and associated professors appointed in 79 specialties evaluated by residents at least once. A total of 491 teachers (54.8%) responded. The structure of the instrument had three sections: sociodemographic and academic data, closed-ended questions with Likert-type scales categorized into 4 themes, and open-ended questions for written comments. Results: More than 85% considered TE as a referent that influences their teaching performance. 93.3% agree that TE helps identify strengths and weaknesses. More than 75% believe that the questionnaire items answered by residents relate to their educational practices. Conclusions: TE is accepted and positively valued by teachers, their responses share the collective imaginary characteristic of higher educational contexts that consider this evaluation as a strategy to provide feedback and improve their teaching performance, and to detect strengths and areas of opportunity. Guided discussions on teaching competencies that go beyond individual reports are required to make visible the academic dynamics in each institution.(AU)


Subject(s)
Humans , Male , Female , Internship and Residency , Education, Graduate , Faculty , Competency-Based Education , Educational Measurement , Evaluation Studies as Topic , Epidemiology, Descriptive , Surveys and Questionnaires
10.
Med Educ Online ; 29(1): 2343205, 2024 Dec 31.
Article in English | MEDLINE | ID: mdl-38626425

ABSTRACT

Shifting to a competency-based (CBME) and not time-bound curricular structure is challenging in the undergraduate medical education (UME) setting for a number of reasons. There are few examples of broad scale CBME-driven interventions that make the UME program less time-bound. However, given the range of student ability and varying speed of acquisition of competencies, this is an area in need of focus. This paper describes a model that uses the macro structure of a UME program to make UME curricula less time-bound, and driven more by student competency acquisition and individual student goals. The 3 + 1 curricular model was derived from the mission of the school, and includes a 3-year core curriculum that all students complete and an individualized phase. Students have an 18 month individualized educational program that meets their developmental needs and their educational and professional goals. This is achieved through a highly structured advising system, including the creation of an Individualized Learning Plan, driven by specific goals and targeted Entrustable Professional Activities (EPA). Students who struggle in achieving core competencies can use individualized time to support competency development and EPA acquisition. For students who have mastered core competencies, options include obtaining a masters degree, clinical immersion, research, and community-based experiences. Students can also graduate after the 3-year core curriculum, and enter residency one year early. Structural approaches such as this may contribute to the norming of the developmental nature of medical education, and can advance culture and systems that support CBME implementation at the UME level.


Subject(s)
Education, Medical, Undergraduate , Education, Medical , Students, Medical , Humans , Schools, Medical , Curriculum , Competency-Based Education , Clinical Competence
11.
Afr J Prim Health Care Fam Med ; 16(1): e1-e5, 2024 Mar 29.
Article in English | MEDLINE | ID: mdl-38572863

ABSTRACT

South Africa is undergoing a significant shift towards implementing enhanced workplace-based assessment methodologies across various specialist training programmes, including family medicine. This paradigm involves the evaluation of Entrustable Professional Activities (EPAs) through comprehensive portfolios of evidence, which a local and national clinical competency committee then assesses. The initial phase of this transformative journey entails the meticulous development of EPAs rooted in discrete units of work. Each EPA delineates the registrar's level of entrustment for autonomous practice, along with the specific supervision requirements. This concise report details the collaborative effort within the discipline of family medicine in South Africa, culminating in the consensus formation of 22 meticulously crafted EPAs for postgraduate family medicine training. The article intricately outlines the systematic structuring and rationale behind the EPAs, elucidating the iterative process employed in their development. Notably, this marks a groundbreaking milestone as the first comprehensive documentation of EPAs nationally for family medicine training in Africa.


Subject(s)
Competency-Based Education , Internship and Residency , Humans , Competency-Based Education/methods , South Africa , Family Practice , Curriculum , Clinical Competence
13.
Chirurgie (Heidelb) ; 95(6): 466-472, 2024 Jun.
Article in German | MEDLINE | ID: mdl-38498122

ABSTRACT

BACKGROUND: Structured competency-based training is one of the most frequently articulated wishes of residents. METHODS: A survey of 19 residents was conducted regarding their satisfaction with the resident education at a level 1 trauma center. In this article the development of a revised competency-based education concept was carried out. RESULTS: The survey reflected uncertainty as to whether the current structures could meet the requirements of the residency regulations. The improved competency-based education concept consists of clinical mentoring, competency-based catalogs of learning objectives, regular theoretical and practical workshops as well as regular and structured staff evaluations. CONCLUSION: The education concept presented reflects the attempt to establish a contemporary surgical training program which will be evaluated as it progresses.


Subject(s)
Competency-Based Education , Education, Medical, Continuing , Internship and Residency , Trauma Centers , Humans , Competency-Based Education/methods , Education, Medical, Continuing/methods , Germany , Surveys and Questionnaires , Clinical Competence/standards , Male , Female , Traumatology/education , Personal Satisfaction , Attitude of Health Personnel , Adult
14.
J Surg Educ ; 81(5): 722-740, 2024 May.
Article in English | MEDLINE | ID: mdl-38492984

ABSTRACT

OBJECTIVES: The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency. DESIGN: A participatory action research model with a mixed methods approach. SETTING: This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department. PARTICIPANTS: Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention. RESULTS: The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4. CONCLUSION: Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.


Subject(s)
Clinical Competence , Competency-Based Education , Education, Medical, Graduate , General Surgery , Education, Medical, Graduate/methods , Competency-Based Education/methods , Humans , General Surgery/education , South Africa , Male , Female , Formative Feedback , Feedback , Teaching , Internship and Residency
15.
Perspect Med Educ ; 13(1): 201-223, 2024.
Article in English | MEDLINE | ID: mdl-38525203

ABSTRACT

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.


Subject(s)
Education, Medical , Medicine , Humans , Competency-Based Education/methods , Education, Medical/methods , Clinical Competence , Publications
16.
BMC Med Educ ; 24(1): 247, 2024 Mar 06.
Article in English | MEDLINE | ID: mdl-38448870

ABSTRACT

BACKGROUND: The introduction of competency-based education models, student centers, and the increased use of formative assessments have led to demands for high-quality test items to be used in assessments. This study aimed to assess the use of an AI tool to generate MCQs type A and evaluate its quality. METHODS: The study design was cross-sectional analytics conducted from June 2023 to August 2023. This study utilized formative TBL. The AI tool (ChatPdf.com) was selected to generate MCQs type A. The generated items were evaluated using a questionnaire for subject experts and an item (psychometric) analysis. The questionnaire to the subject experts about items was formed based on item quality and rating of item difficulty. RESULTS: The total number of recurrent staff members as experts was 25, and the questionnaire response rate was 68%. The quality of the items ranged from good to excellent. None of the items had scenarios or vignettes and were direct. According to the expert's rating, easy items represented 80%, and only two had moderate difficulty (20%). Only one item out of the two moderate difficulties had the same difficulty index. The total number of students participating in TBL was 48. The mean mark was 4.8 ± 1.7 out of 10. The KR20 is 0.68. Most items were of moderately difficult (90%) and only one was difficult (10%). The discrimination index of the items ranged from 0.77 to 0.15. Items with excellent discrimination represented 50% (5), items with good discrimination were 3 (30%), and only one time was poor (10%), and one was none discriminating. The non-functional distractors were 26 (86.7%), and the number of non-functional distractors was four (13.3%). According to distractor analysis, 60% of the items were excellent, and 40% were good. A significant correlation (p = 0.4, r = 0.30) was found between the difficulty and discrimination indices. CONCLUSION: Items constructed using AI had good psychometric properties and quality, measuring higher-order domains. AI allows the construction of many items within a short time. We hope this paper brings the use of AI in item generation and the associated challenges into a multi-layered discussion that will eventually lead to improvements in item generation and assessment in general.


Subject(s)
Education, Medical , Humans , Cross-Sectional Studies , Competency-Based Education , Psychometrics , Research Design
17.
J Surg Educ ; 81(5): 741-752, 2024 May.
Article in English | MEDLINE | ID: mdl-38553368

ABSTRACT

OBJECTIVE: The purpose of this qualitative study was to examine responses related to entrustment and feedback comments from an assessment tool. DESIGN: Qualitative analyses using semi-structured interviews and analysis of narrative comments. SETTING: Main hospital OR suite at a large academic medical center. PARTICIPANTS: faculty, and residents who work in the OR suite. RESULTS: Seven of the 14 theoretical domains from the Theoretical Domains Framework were identified as influencing faculty decision on entrustment: knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. The majority (651/1116 (58.4%)) of faculty comments were critical/modest praise and relevant, consistent across all 6 EPAs. The written in feedback comments for all 1,116 Web App EPA assessments yielded a total of 1,599 sub-competency specific responses. These responses were mapped to core competencies, and at least once to 13 of the 23 ACGME subcompetencies. CONCLUSIONS: Domains identified as influencing faculty decision on entrustment were knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. Most narrative feedback comments were critical/modest praise and relevant, consistent across each of the EPAs.


Subject(s)
Anesthesiology , Clinical Competence , Faculty, Medical , Internship and Residency , Humans , Anesthesiology/education , Qualitative Research , Female , Male , Education, Medical, Graduate/methods , Competency-Based Education/methods , Decision Making , Feedback
18.
J Surg Educ ; 81(5): 626-638, 2024 May.
Article in English | MEDLINE | ID: mdl-38555246

ABSTRACT

PURPOSE: The Accreditation Council for Graduate Medical Education (ACGME) introduced General Surgery Milestones 1.0 in 2014 and Milestones 2.0 in 2020 as steps toward competency-based training. Analysis will inform residency programs on curriculum development, assessment, feedback, and faculty development. This study describes the distributions and trends for Milestones 1.0 and 2.0 ratings and proportion of residents not achieving the level 4.0 graduation target. METHODS: A deidentified dataset of milestone ratings for all ACGME-accredited General Surgery residency programs in the United States was used. Medians and interquartile ranges (IQR) were reported for milestone ratings at each PGY level. Percentages of PGY-5s receiving final year ratings of less than 4.0 were calculated. Wilcoxon rank sum tests were used to compare 1.0 and 2.0 median ratings. Kruskal-Wallis and Bonferroni post-hoc tests were used to compare median ratings across time periods and PGY levels. Chi-squared tests were used to compare the proportion of level 4.0 nonachievement under both systems. RESULTS: Milestones 1.0 data consisted of 13,866 residents and Milestones 2.0 data consisted of 7,633 residents. For 1.0 and 2.0, all competency domain median ratings were higher for subsequent years of training. Milestones 2.0 had significantly higher median ratings at all PGY levels for all competency domains except Medical Knowledge. Percentages of PGY-5 residents not achieving the graduation target in Milestones 1.0 ranged from 27% to 42% and in 2.0 from 5% to 13%. For Milestones 1.0, all subcompetencies showed an increased number of residents achieving the graduation target from 2014 to 2019. CONCLUSIONS: This study of General Surgery Milestones 1.0 and 2.0 data uncovered significant increases in average ratings and significantly fewer residents not achieving the graduation target under the 2.0 system. We hypothesize that these findings may be related more to rating bias given the change in rating scales, rather than a true increase in resident ability.


Subject(s)
Accreditation , Clinical Competence , Education, Medical, Graduate , General Surgery , Internship and Residency , General Surgery/education , United States , Humans , Competency-Based Education , Time Factors , Male
19.
BMC Med Educ ; 24(1): 297, 2024 Mar 15.
Article in English | MEDLINE | ID: mdl-38491437

ABSTRACT

BACKGROUND: Recently, all medical universities in Sweden jointly developed a framework for Entrustable Professional Activities (EPAs) for work-based training and assessment. This framework is now being introduced nationally in the new 6-year undergraduate medical programme that directly lead to a licence to practise. When EPAs are introduced, it is of central importance to gain clinical supervisors' acceptance to apply the framework in their supervision of students. The aim of this study was therefore to investigate how clinical supervisors, not familiar with EPAs, experience clinical supervision using the framework for EPAs. METHODS: We used a purposive sampling to recruit clinical supervisors. They were given written information on EPAs with a selection of suitable EPAs and the Swedish observation rating scale for assessment of autonomy, and they were offered to attend a 30-minute introductory web course. The participants were informed that EPAs were to be tested, and the students were asked to participate. After the study period the clinical supervisors participated in semi-structured interviews. Inductive qualitative content analysis was used to analyse the transcribed interviews. RESULTS: Three general themes emerged in the qualitative analysis: Promoting Feedback, Trusting Assessments and Engaging Stakeholders. The participants described benefits from using EPAs, but pointed out a need for preparation and adaptation to facilitate implementation. The structure was perceived to provide structured support for feedback, student involvement, entrustment decisions, enabling supervisors to allow the students to do more things independently, although some expressed caution to rely on others' assessments. Another concern was whether assessments of EPAs would be perceived as a form of examination, steeling focus from formative feedback. To understand the concept of EPA, the short web-based course and written information was regarded as sufficient. However, concern was expressed whether EPA could be applied by all clinical supervisors. Involvement and adaption of the workplace was pointed out as important since more frequent observation and feedback, with documentation requirements, increase the time required for supervision. CONCLUSIONS: EPAs were accepted as beneficial, promoting structured feedback and assessments of the students' autonomy. Preparation of supervisors and students as well as involvement and adaptation of the workplace was pointed out as important.


Subject(s)
Internship and Residency , Students, Medical , Humans , Competency-Based Education , Pilot Projects , Sweden , Preceptorship , Clinical Competence
20.
Can J Surg ; 67(2): E99-E107, 2024.
Article in English | MEDLINE | ID: mdl-38453348

ABSTRACT

BACKGROUND: General surgeons play an important role in the provision of trauma care in Canada and the current extent of their trauma experience during training is unknown. We sought to quantify the operative and nonoperative educational experiences among Canadian general surgery trainees. METHODS: We conducted a multicentre retrospective study of major operative exposures experienced by general surgery residents, as identified using institutional trauma registries and subsequent chart-level review, for 2008-2018. We also conducted a site survey on trauma education and structure. RESULTS: We collected data on operative exposure for general surgery residents from 7 programs and survey data from 10 programs. Operations predominantly occurred after hours (73% after 1700 or on weekends) and general surgery residents were absent from a substantial proportion (25%) of relevant trauma operations. The structure of trauma education was heterogeneous among programs, with considerable site-specific variability in the involvement of surgical specialties in trauma care. During their training, graduating general surgery residents each experienced around 4 index trauma laparotomies, 1 splenectomy, 1 thoracotomy, and 0 neck explorations for trauma. CONCLUSION: General surgery residents who train in Canada receive variable and limited exposure to operative and nonoperative trauma care. These data can be used as a baseline to inform the application of competency-based medical education in trauma care for general surgery training in Canada.


Subject(s)
General Surgery , Internship and Residency , Humans , Retrospective Studies , Canada , Competency-Based Education , Registries , Clinical Competence , General Surgery/education , Education, Medical, Graduate
SELECTION OF CITATIONS
SEARCH DETAIL
...