RÉSUMÉ
Standardised proficiency-based progression is the cornerstone of safe robotic skills acquisition, however, is currently lacking within surgical training curricula. Expert consensuses have defined a modular pathway to accredit surgeons. This study aimed to address the lack of a formal, pre-clinical core robotic skills, proficiency-based accreditation curriculum in the UK. Novice robotic participants underwent a four-day pre-clinical core robotic skills curriculum incorporating multimodal assessment. Modifiable-Global Evaluative Assessment of Robotic Skills (M-GEARS), VR-automated performance metrics (APMs) and Objective Clinical Human Reliability Analysis (OCHRA) error methodology assessed performance at the beginning and end of training. Messick's validity concept and a curriculum evaluation model were utilised. Feedback was collated. Proficiency-based progression, benchmarking, tool validity and reliability was assessed through comparative and correlational statistical methods. Forty-seven participants were recruited. Objective assessment of VR and dry models across M-GEARS, APMs and OCHRA demonstrated significant improvements in technical skill (p < 0.001). Concurrent validity between assessment tools demonstrated strong correlation in dry and VR tasks (r = 0.64-0.92, p < 0.001). OCHRA Inter-rater reliability was excellent (r = 0.93, p < 0.001 and 81% matched error events). A benchmark was established with M-GEARS and for the curriculum at 80%. Thirty (63.82%) participants passed. Feedback was 5/5 stars on average, with 100% recommendation. Curriculum evaluation fulfilled all five domains of Messick's validity. Core robotic surgical skills training can be objectively evaluated and benchmarked to provide accreditation in basic robotic skills. A strategy is necessary to enrol standardised curricula into national surgical training at an early stage to ensure patient safety.
Sujet(s)
Agrément , Compétence clinique , Programme d'études , Interventions chirurgicales robotisées , Agrément/normes , Interventions chirurgicales robotisées/enseignement et éducation , Interventions chirurgicales robotisées/normes , Humains , Royaume-Uni , Compétence clinique/normes , Reproductibilité des résultats , Mâle , FemelleRÉSUMÉ
BACKGROUND: An assessment program should be inclusive and ensure that the various components of medical knowledge, clinical skills, and professionalism are assessed. The level and the variation over time in the strength of the correlation between these components of assessment is still a matter of study. Based on the meaningful learning theory and the integrated learning theory, we hypothesize that these components increase their connections during the medical school course. METHODS: This is a retrospective cohort study that analyzed data collected for a 10-year period in one medical school. We included students from the 3rd to 6th year of medical school from 2011 to 2021. Three assessment components were addressed: Knowledge, Clinical Skills, and Professionalism. For data analysis, Pearson correlation coefficients (R) and R2 were calculated to study the correlation between variables and a z-test on Fisher's r-to-z was used to determine the differences between correlation coefficients. RESULTS: 949 medical students were included in the study. The correlation between Clinical Skills and Professionalism showed a medium to strong association (Pearson's R ranging from 0.485 to 0.734), while the correlation between Knowledge and Professionalism was weaker but exhibited a steady evolution with Pearson's R fluctuating between 0.075 and 0.218. The Knowledge and Clinical Skills correlation became statistically significant from 2013 onwards and peaking at Pearson's R of 0.440 for the cohort spanning 2016-2019. We also revealed a strengthening of correlations between Professionalism and Clinical Skills from the beginning to the end of clinical training, but not with the knowledge component. CONCLUSIONS: This analysis contributes to our understanding of the dynamics of correlations of different assessment components within an institution and provides a framework for how they interact and influence each other. TRIAL REGISTRATION: This study was not a clinical trial, but a retrospective observational study, without health care interventions. Nevertheless, we provide herein the number of the study as submitted to the Ethics committee - CEICVS 146/2021.
Sujet(s)
Compétence clinique , Évaluation des acquis scolaires , Professionnalisme , Écoles de médecine , Étudiant médecine , Humains , Études rétrospectives , Compétence clinique/normes , Professionnalisme/normes , Enseignement médical premier cycle/normes , Femelle , Mâle , Études longitudinalesRÉSUMÉ
The objective of academic training is to prepare midwives as independent healthcare professionals to make a substantial contribution to the healthcare of women in their reproductive years as well as to the health of their children and families. This article therefore describes the professional and educational requirements derived from the legal midwifery competencies within the new midwifery act. Furthermore, it identifies the conditions that need to be established to enable midwives in Germany to practise to their full scope in compliance with statutory responsibilities. Educational science, academic efforts, policymaking and accompanying research should work in synergy. This in turn enables midwives to achieve the maximum scope of their skills, with the objective of promoting physiological pregnancies and births. Consequently, it can strengthen early parenthood in alignment with the national health objectives of "health around childbirth". The academisation of the midwifery profession presents a profound opportunity for professional development in Germany. It is essential that midwives receive training based on the principles of educational science and care structures that are yet to be developed. This can enable them to perform within the wide range of their professional tasks to the highest standards, thereby ensuring the optimal care of their clients. Moreover, there is a chance to implement sustainable improvements in healthcare provision for women and their families during the reproductive phase and the period of parenthood in Germany.
Sujet(s)
Profession de sage-femme , Allemagne , Profession de sage-femme/enseignement et éducation , Humains , Compétence clinique/normes , Femelle , Programme d'études/normes , Programme d'études/tendances , GrossesseRÉSUMÉ
Background: As part of the MERLIN project (Medical Education Research - Lehrforschung im Netz BW), funded by the Federal Ministry of Education and Research, graduate surveys were carried out at the Medical Faculty of Freiburg from 2012-2020. This article will primarily address the question of how the study conditions and competence orientation in Freiburg are assessed and where there is still a need for optimization. Method: The surveys were conducted among graduates of human medicine at the Freiburg Medical Faculty 1.5 years after graduation. Participation was possible using paper and online questionnaires. The response rates were 36%-43%. Results: The study conditions were largely rated as good. There is a need for optimization, especially in the area of scientific work. The level of skills acquired was assessed as good to moderate. There were discrepancies between the level of competence achieved during the course of study and the level of competence required to start a career. Discussion: There is a need for development in terms of preparation for starting a career. Compared to the professionally required level of competence, self-assessment was worse in most competence domains. In Freiburg there are approaches to further promote the acquisition of skills during studies. In order to evaluate these developments and future changes in the context of studies, graduate surveys are relevant. Conclusion: Graduate surveys are suitable for generating data on the basis of which curriculum design can be carried out or which can be used to prepare for change processes. The surveys in Freiburg will therefore be continued and supplemented with new, needs-based questions.
Sujet(s)
Compétence clinique , Humains , Enquêtes et questionnaires , Compétence clinique/normes , Allemagne , Enseignement médical , Programme d'études/normes , Femelle , MâleRÉSUMÉ
The current situation in Germany is characterised by significant differences between the two types of higher education institutions offering bachelor's degree programmes in midwifery at both universities of applied sciences and universities. These differences are noticeable in admission procedures, resource allocation, content focus and competence assessment at the respective institutions, which in turn result in heterogeneous study experiences. This article highlights the challenges currently facing bachelor degree programmes and the academic qualification of midwives, and identifies future requirements for the development of degree programmes in theory and practice as well as theory-practice transfer, and assessment formats. Furthermore, this article covers the content-related and structural-organisational requirements to develop in-depth academic skills grounded in theory teaching, the facilitation of clinical placements at an academic level, the training of qualified practical instructors and the development of applicable competence-based assessment formats, especially for the state exam. The development of a standardised, high-quality academic education for midwives in Germany requires networking of the different academic sites/locations to exchange experiences in teaching/learning and assessment formats. Furthermore, it can facilitate the development of a standardised competence-oriented model and core curriculum as well as the definition of quality criteria and standards for study programmes of midwifery science. The Midwifery Science Committee (AHW) in the DACH Assoviation for Medical Education (GMA) offers an optimal platform for cooperation between the different universities. The existing challenges for the further professional development of midwives can only be overcome by collaboration and pooled expertise.
Sujet(s)
Programme d'études , Profession de sage-femme , Allemagne , Profession de sage-femme/enseignement et éducation , Humains , Programme d'études/normes , Programme d'études/tendances , Formation au diplôme infirmier (USA)/méthodes , Compétence clinique/normes , Évaluation des acquis scolaires/méthodesRÉSUMÉ
Objectives: Teaching communication skills plays a pivotal role in medical curricula. The aim of this article is to describe and evaluate a new communication curriculum developed at the Faculty of Medicine, University of Augsburg (KomCuA), which was conceptualized by an interdisciplinary team based on recommended quality standards (i.e., helical, integrated, longitudinal). Methods: A total of 150 medical students enrolled in the 1st, 3rd, and ≥5th semester participated in the study. They completed an online survey (numerical rating scales and validated questionnaires) evaluating their current communication skills, how these developed across the curriculum in terms of quality and self-confidence, and how helpful they considered practicing in small group tutorials with simulated patients. The students' attitudes towards communication and empathy in the context of medical care were additionally assessed. The students' responses were compared across semesters using one-way univariate analysis of variance (ANOVA). Results: Overall, students reported improved communications skills due to attending the KomCuA and further considered practicing with simulated patients as being very helpful (large effect sizes). Compared to 1st semester students, 3rd and ≥5th semester students reported better communication skills (medium to large effect sizes). Additionally, ≥5th semester students showed stronger agreement towards the relevance of empathy in the context of medical care (medium effect size) compared to both 1st and 3rd semester students. Conclusion: The KomCuA has shown to be an effective communication curriculum to support medical students in the development of their communication skills and positive attitudes towards empathy. Additional studies assessing students' communication skills and empathic attitudes longitudinally are warranted to confirm the present results and to gain further knowledge on how these essential skills and attitudes develop across medical curricula.
Sujet(s)
Communication , Programme d'études , Enseignement médical premier cycle , Étudiant médecine , Humains , Étudiant médecine/psychologie , Enseignement médical premier cycle/méthodes , Mâle , Enquêtes et questionnaires , Femelle , Empathie , Relations médecin-patient , Allemagne , Compétence clinique/normes , AdulteRÉSUMÉ
Background: With the increasing application of large language models like ChatGPT in various industries, its potential in the medical domain, especially in standardized examinations, has become a focal point of research. Objective: The aim of this study is to assess the clinical performance of ChatGPT, focusing on its accuracy and reliability in the Chinese National Medical Licensing Examination (CNMLE). Methods: The CNMLE 2022 question set, consisting of 500 single-answer multiple choices questions, were reclassified into 15 medical subspecialties. Each question was tested 8 to 12 times in Chinese on the OpenAI platform from April 24 to May 15, 2023. Three key factors were considered: the version of GPT-3.5 and 4.0, the prompt's designation of system roles tailored to medical subspecialties, and repetition for coherence. A passing accuracy threshold was established as 60%. The χ2 tests and κ values were employed to evaluate the model's accuracy and consistency. Results: GPT-4.0 achieved a passing accuracy of 72.7%, which was significantly higher than that of GPT-3.5 (54%; P<.001). The variability rate of repeated responses from GPT-4.0 was lower than that of GPT-3.5 (9% vs 19.5%; P<.001). However, both models showed relatively good response coherence, with κ values of 0.778 and 0.610, respectively. System roles numerically increased accuracy for both GPT-4.0 (0.3%-3.7%) and GPT-3.5 (1.3%-4.5%), and reduced variability by 1.7% and 1.8%, respectively (P>.05). In subgroup analysis, ChatGPT achieved comparable accuracy among different question types (P>.05). GPT-4.0 surpassed the accuracy threshold in 14 of 15 subspecialties, while GPT-3.5 did so in 7 of 15 on the first response. Conclusions: GPT-4.0 passed the CNMLE and outperformed GPT-3.5 in key areas such as accuracy, consistency, and medical subspecialty expertise. Adding a system role insignificantly enhanced the model's reliability and answer coherence. GPT-4.0 showed promising potential in medical education and clinical practice, meriting further study.
Sujet(s)
Évaluation des acquis scolaires , Autorisation d'exercer la médecine , Humains , Chine , Évaluation des acquis scolaires/méthodes , Évaluation des acquis scolaires/normes , Reproductibilité des résultats , Compétence clinique/normesRÉSUMÉ
BACKGROUND: Evaluating the accuracy and educational utility of artificial intelligence-generated medical cases, especially those produced by large language models such as ChatGPT-4 (developed by OpenAI), is crucial yet underexplored. OBJECTIVE: This study aimed to assess the educational utility of ChatGPT-4-generated clinical vignettes and their applicability in educational settings. METHODS: Using a convergent mixed methods design, a web-based survey was conducted from January 8 to 28, 2024, to evaluate 18 medical cases generated by ChatGPT-4 in Japanese. In the survey, 6 main question items were used to evaluate the quality of the generated clinical vignettes and their educational utility, which are information quality, information accuracy, educational usefulness, clinical match, terminology accuracy (TA), and diagnosis difficulty. Feedback was solicited from physicians specializing in general internal medicine or general medicine and experienced in medical education. Chi-square and Mann-Whitney U tests were performed to identify differences among cases, and linear regression was used to examine trends associated with physicians' experience. Thematic analysis of qualitative feedback was performed to identify areas for improvement and confirm the educational utility of the cases. RESULTS: Of the 73 invited participants, 71 (97%) responded. The respondents, primarily male (64/71, 90%), spanned a broad range of practice years (from 1976 to 2017) and represented diverse hospital sizes throughout Japan. The majority deemed the information quality (mean 0.77, 95% CI 0.75-0.79) and information accuracy (mean 0.68, 95% CI 0.65-0.71) to be satisfactory, with these responses being based on binary data. The average scores assigned were 3.55 (95% CI 3.49-3.60) for educational usefulness, 3.70 (95% CI 3.65-3.75) for clinical match, 3.49 (95% CI 3.44-3.55) for TA, and 2.34 (95% CI 2.28-2.40) for diagnosis difficulty, based on a 5-point Likert scale. Statistical analysis showed significant variability in content quality and relevance across the cases (P<.001 after Bonferroni correction). Participants suggested improvements in generating physical findings, using natural language, and enhancing medical TA. The thematic analysis highlighted the need for clearer documentation, clinical information consistency, content relevance, and patient-centered case presentations. CONCLUSIONS: ChatGPT-4-generated medical cases written in Japanese possess considerable potential as resources in medical education, with recognized adequacy in quality and accuracy. Nevertheless, there is a notable need for enhancements in the precision and realism of case details. This study emphasizes ChatGPT-4's value as an adjunctive educational tool in the medical field, requiring expert oversight for optimal application.
Sujet(s)
Intelligence artificielle , Humains , Japon , Mâle , Femelle , Enquêtes et questionnaires , Adulte , Enseignement médical/méthodes , Compétence clinique/normes , Peuples d'Asie de l'EstRÉSUMÉ
BACKGROUND: Individuals with intellectual disabilities experience barriers to quality healthcare. To reduce this disparity, equipping medical trainees with the knowledge and skills required for treating this patient population is critical. Our aim is to describe the breadth of instructional interventions and identify gaps in intellectual disability medical education curricula. METHOD: Using scoping review methods, the intellectual disability programmes described in 27 articles were evaluated and their coverage of the six core competencies on disability for health care education was examined. RESULTS: The most frequently represented core competencies were disability conceptual frameworks, professionalism and communication, and clinical assessment, which were, in most programmes, fulfilled by activities involving individuals with intellectual disabilities. Uneven competency coverage warrants consideration. CONCLUSIONS: Considerable variabilities exist in medical school curricula on intellectual disabilities. Using core competencies on disability for health care education for curricular design and evaluation would provide a coherent training experience in this important area.
Sujet(s)
Compétence clinique , Programme d'études , Déficience intellectuelle , Étudiant médecine , Humains , Déficience intellectuelle/rééducation et réadaptation , Compétence clinique/normes , Enseignement médicalRÉSUMÉ
Importance: Residents must prepare for effective communication with patients after medical errors. The video-based communication assessment (VCA) is software that plays video of a patient scenario, asks the physician to record what they would say, engages crowdsourced laypeople to rate audio recordings of physician responses, and presents feedback to physicians. Objective: To evaluate the effectiveness of VCA feedback in resident error disclosure skill training. Design, Setting, and Participants: This single-blinded, randomized clinical trial was conducted from July 2022 to May 2023 at 7 US internal medicine and family medicine residencies (10 total sites). Participants were second-year residents attending required teaching conferences. Data analysis was performed from July to December 2023. Intervention: Residents completed 2 VCA cases at time 1 and were randomized to the intervention, an individual feedback report provided in the VCA application after 2 weeks, or to control, in which feedback was not provided until after time 2. Residents completed 2 additional VCA cases after 4 weeks (time 2). Main Outcomes and Measures: Panels of crowdsourced laypeople rated recordings of residents disclosing simulated medical errors to create scores on a 5-point scale. Reports included learning points derived from layperson comments. Mean time 2 ratings were compared to test the hypothesis that residents who had access to feedback on their time 1 performance would score higher at time 2 than those without feedback access. Residents were surveyed about demographic characteristics, disclosure experience, and feedback use. The intervention's effect was examined using analysis of covariance. Results: A total of 146 residents (87 [60.0%] aged 25-29 years; 60 female [41.0%]) completed the time 1 VCA, and 103 (70.5%) completed the time 2 VCA (53 randomized to intervention and 50 randomized to control); of those, 28 (54.9%) reported reviewing their feedback. Analysis of covariance found a significant main effect of feedback between intervention and control groups at time 2 (mean [SD] score, 3.26 [0.45] vs 3.14 [0.39]; difference, 0.12; 95% CI, 0.08-0.48; P = .01). In post hoc comparisons restricted to residents without prior disclosure experience, intervention residents scored higher than those in the control group at time 2 (mean [SD] score, 3.33 [0.43] vs 3.09 [0.44]; difference, 0.24; 95% CI, 0.01-0.48; P = .007). Worse performance at time 1 was associated with increased likelihood of dropping out before time 2 (odds ratio, 2.89; 95% CI, 1.06-7.84; P = .04). Conclusions and Relevance: In this randomized clinical trial, self-directed review of crowdsourced feedback was associated with higher ratings of internal medicine and family medicine residents' error disclosure skill, particularly for those without real-life error disclosure experience, suggesting that such feedback may be an effective way for residency programs to address their requirement to prepare trainees for communicating with patients after medical harm. Trial Registration: ClinicalTrials.gov Identifier: NCT06234085.
Sujet(s)
Externalisation ouverte , Internat et résidence , Erreurs médicales , Humains , Internat et résidence/méthodes , Femelle , Mâle , Externalisation ouverte/méthodes , Adulte , Erreurs médicales/prévention et contrôle , Compétence clinique/statistiques et données numériques , Compétence clinique/normes , Méthode en simple aveugle , Révélation de la vérité , Médecine interne/enseignement et éducation , Relations médecin-patient , RétroactionRÉSUMÉ
BACKGROUND: Although history taking is fundamental for diagnosing medical conditions, teaching and providing feedback on the skill can be challenging due to resource constraints. Virtual simulated patients and web-based chatbots have thus emerged as educational tools, with recent advancements in artificial intelligence (AI) such as large language models (LLMs) enhancing their realism and potential to provide feedback. OBJECTIVE: In our study, we aimed to evaluate the effectiveness of a Generative Pretrained Transformer (GPT) 4 model to provide structured feedback on medical students' performance in history taking with a simulated patient. METHODS: We conducted a prospective study involving medical students performing history taking with a GPT-powered chatbot. To that end, we designed a chatbot to simulate patients' responses and provide immediate feedback on the comprehensiveness of the students' history taking. Students' interactions with the chatbot were analyzed, and feedback from the chatbot was compared with feedback from a human rater. We measured interrater reliability and performed a descriptive analysis to assess the quality of feedback. RESULTS: Most of the study's participants were in their third year of medical school. A total of 1894 question-answer pairs from 106 conversations were included in our analysis. GPT-4's role-play and responses were medically plausible in more than 99% of cases. Interrater reliability between GPT-4 and the human rater showed "almost perfect" agreement (Cohen κ=0.832). Less agreement (κ<0.6) detected for 8 out of 45 feedback categories highlighted topics about which the model's assessments were overly specific or diverged from human judgement. CONCLUSIONS: The GPT model was effective in providing structured feedback on history-taking dialogs provided by medical students. Although we unraveled some limitations regarding the specificity of feedback for certain feedback categories, the overall high agreement with human raters suggests that LLMs can be a valuable tool for medical education. Our findings, thus, advocate the careful integration of AI-driven feedback mechanisms in medical training and highlight important aspects when LLMs are used in that context.
Sujet(s)
Recueil de l'anamnèse , Simulation sur patients standardisés , Étudiant médecine , Humains , Études prospectives , Recueil de l'anamnèse/méthodes , Recueil de l'anamnèse/normes , Étudiant médecine/psychologie , Femelle , Mâle , Compétence clinique/normes , Intelligence artificielle , Rétroaction , Reproductibilité des résultats , Enseignement médical premier cycle/méthodesRÉSUMÉ
Background: Teaching medical students the skills required to acquire, interpret, apply, and communicate clinical information is an integral part of medical education. A crucial aspect of this process involves providing students with feedback regarding the quality of their free-text clinical notes. Objective: The goal of this study was to assess the ability of ChatGPT 3.5, a large language model, to score medical students' free-text history and physical notes. Methods: This is a single-institution, retrospective study. Standardized patients learned a prespecified clinical case and, acting as the patient, interacted with medical students. Each student wrote a free-text history and physical note of their interaction. The students' notes were scored independently by the standardized patients and ChatGPT using a prespecified scoring rubric that consisted of 85 case elements. The measure of accuracy was percent correct. Results: The study population consisted of 168 first-year medical students. There was a total of 14,280 scores. The ChatGPT incorrect scoring rate was 1.0%, and the standardized patient incorrect scoring rate was 7.2%. The ChatGPT error rate was 86%, lower than the standardized patient error rate. The ChatGPT mean incorrect scoring rate of 12 (SD 11) was significantly lower than the standardized patient mean incorrect scoring rate of 85 (SD 74; P=.002). Conclusions: ChatGPT demonstrated a significantly lower error rate compared to standardized patients. This is the first study to assess the ability of a generative pretrained transformer (GPT) program to score medical students' standardized patient-based free-text clinical notes. It is expected that, in the near future, large language models will provide real-time feedback to practicing physicians regarding their free-text notes. GPT artificial intelligence programs represent an important advance in medical education and medical practice.
Sujet(s)
Étudiant médecine , Humains , Études rétrospectives , Enseignement médical premier cycle/méthodes , Évaluation des acquis scolaires/méthodes , Langage , Recueil de l'anamnèse/méthodes , Recueil de l'anamnèse/normes , Compétence clinique/normes , MâleRÉSUMÉ
BACKGROUND AND OBJECTIVES: The burden of disease for Australian children from non-acute conditions is growing; however, little is known about how well prevocational training experiences prepare trainee doctors. This study examines the confidence of general practice registrars in managing paediatric consultations in primary care and whether confidence varies by prevocational training type. METHOD: This was a cross-sectional national survey of Australian general practice registrars that measured confidence in managing paediatric primary care presentations. RESULTS: Respondents reported feeling confident (65%) or very confident (8%) in managing paediatrics in primary care, with higher confidence for those more advanced in their training or with greater exposure to paediatrics during prevocational training. Regression models showed registrars were more likely to report higher confidence when managing acute versus non-acute presentations. DISCUSSION: Although most registrars reported confidence in managing paediatric presentations, confidence levels were notably lower for non-acute conditions and when prevocational training experiences included limited exposure to paediatric patients.
Sujet(s)
Pédiatrie , Soins de santé primaires , Humains , Études transversales , Australie , Soins de santé primaires/statistiques et données numériques , Mâle , Femelle , Pédiatrie/méthodes , Pédiatrie/statistiques et données numériques , Enquêtes et questionnaires , Compétence clinique/statistiques et données numériques , Compétence clinique/normes , Adulte , Médecine générale/méthodes , Médecine générale/statistiques et données numériques , Enfant , Adulte d'âge moyenRÉSUMÉ
Designing a transition to practice program to increase foundational skill competency and interprofessional collaboration before unit orientation can produce outcomes that consistently outperform benchmarks. Established structures and processes drive the program to be easily amenable to incremental improvement, further maturing and sustaining the program's positive impacts. [J Contin Educ Nurs. 2024;55(8):369-371.].
Sujet(s)
Compétence clinique , Programme d'études , Formation continue infirmier , Humains , Mâle , Femelle , Adulte , Formation continue infirmier/organisation et administration , Adulte d'âge moyen , Compétence clinique/normes , Mise au point de programmes , Personnel infirmier hospitalier/enseignement et éducation , Évaluation de programme , Relations interprofessionnellesRÉSUMÉ
BACKGROUND: The integration of Health System Science (HSS) in medical education emphasizes mastery of competencies beyond mere knowledge acquisition. With the shift to online platforms during the COVID-19 pandemic, there is an increased emphasis on Technology Enhanced Assessment (TEA) methods, such as video assessments, to evaluate these competencies. This study investigates the efficacy of online video assessments in evaluating medical students' competency in HSS. METHODS: A comprehensive assessment was conducted on first-year medical students (n = 10) enrolled in a newly developed curriculum integrating Health System Science (HSS) into the Bachelor of Medicine program in 2021. Students undertook three exams focusing on HSS competency. Their video responses were evaluated by a panel of seven expert assessors using a detailed rubric. Spearman rank correlation and the Interclass Correlation Coefficient (ICC) were utilized to determine correlations and reliability among assessor scores, while a mixed-effects model was employed to assess the relationship between foundational HSS competencies (C) and presentation skills (P). RESULTS: Positive correlations were observed in inter-rater reliability, with ICC values suggesting a range of reliability from poor to moderate. A positive correlation between C and P scores was identified in the mixed-effects model. The study also highlighted variations in reliability and correlation, which might be attributed to differences in content, grading criteria, and the nature of individual exams. CONCLUSION: Our findings indicate that effective presentation enhances the perceived competency of medical students, emphasizing the need for standardized assessment criteria and consistent assessor training in online environments. This study highlights the critical roles of comprehensive competency assessments and refined presentation skills in online medical education, ensuring accurate and reliable evaluations.
Sujet(s)
COVID-19 , Compétence clinique , Programme d'études , Évaluation des acquis scolaires , Enregistrement sur magnétoscope , Humains , Évaluation des acquis scolaires/méthodes , Compétence clinique/normes , Enseignement médical premier cycle/normes , Étudiant médecine , Reproductibilité des résultats , Enseignement à distance , SARS-CoV-2 , MâleRÉSUMÉ
BACKGROUND: Implementation support practitioners (ISPs) are professionals that support others to implement evidence-informed practices, programs, and policies in various service delivery settings to achieve population outcomes. Measuring the use of competencies by ISPs provides a unique opportunity to assess an understudied facet of implementation science-how knowledge, attitudes, and skills used by ISPs affects sustainable change in complicated and complex service systems. This study describes the development and validation of a measure-the Implementation Support Competencies Assessment (ISCA)-that assesses implementation support competencies, with versatile applications across service contexts. METHODS: Recently developed practice guide materials included operationalizations of core competencies for ISPs across three domains: co-creation and engagement, ongoing improvement, and sustaining change. These operationalizations, in combination with recent empirical and conceptual work, provided an initial item pool and foundation on which to advance measurement development, largely from a confirmatory perspective (as opposed to exploratory). The measure was further refined through modified cognitive interviewing with three highly experienced ISPs and pilot-testing with 39 individuals enrolled in a university-based certificate program in implementation practice. To recruit a sample for validation analyses, we leveraged a listserv of nearly 4,000 individuals who have registered for or expressed interest in various events and trainings focused on implementation practice offered by an implementation science collaborative housed within a research-intensive university in the Southeast region of the United States. Our final analytic sample included 357 participants who self-identified as ISPs. RESULTS: Assessments of internal consistency reliability for each competency-specific item set yielded evidence of strong reliability. Results from confirmatory factor analyses provided evidence for the factorial and construct validity of all three domains and associated competencies in the ISCA. CONCLUSIONS: The findings suggest that one's possession of high levels of competence across each of the three competency domains is strongly associated with theorized outcomes that can promote successful and sustainable implementation efforts among those who receive implementation support from an ISP. The ISCA serves as a foundational tool for workforce development to formally measure and assess improvement in the skills that are required to tailor a package of implementation strategies situated in context.
Sujet(s)
Science de la mise en oeuvre , Psychométrie , Humains , Reproductibilité des résultats , Femelle , Mâle , Compétence professionnelle/normes , Adulte , Compétence clinique/normes , Enquêtes et questionnaires/normes , Adulte d'âge moyenRÉSUMÉ
Background: The responsibility for addressing the healthcare needs of PWUS is the responsibility of all physicians. Within the healthcare system, research consistently reveals inequitable experiences in healthcare with people who use substances (PWUS) reporting stigmatization, marginalization, and a lack of compassion. Objectives: The aim of this scoping review was to find and describe competencies being taught, developed, and fostered within medical education and then to provide recommendations to improve care for this population of patients. Results: Nineteen articles were included. Recommended knowledge competencies tend to promote understanding neurophysiological changes caused by substances, alongside knowing how to evaluate of 'risky' behaviours. Commonly recommended skills relate to the screening and management of substance use disorders. Recommended attitude competencies include identifying personal bias and establishing a patient-centered culture among practice teams. The disease model of addiction informed all papers, with no acknowledgement of potential beneficial or non-problematic experiences of substance use. To enhance knowledge-type competencies, medical education programs are advised to include addiction specialists as educators and prevent stigmatization through the hidden curriculum. Conclusion: To reduce experiences of stigmatization and marginalization among patients who use illicit substances and to improve quality of care, knowledge, skills, and attitudes competencies can be more effectively taught in medical education programs.Résumé.
Contexte: Il incombe à tous les médecins de répondre aux besoins en matière de soins de santé des consommateurs de substances psychoactives. Au sein du système de santé, les recherches révèlent régulièrement des expériences inéquitables en matière de soins de santé, les consommateurs de substances faisant état de stigmatisation, de marginalisation et d'un manque de compassion. Objectifs: L'objectif de cette étude exploratoire était de trouver et de décrire les compétences enseignées, développées et encouragées dans le cadre de la formation médicale, puis de formuler des recommandations pour améliorer les soins prodigués à ces patients. Résultats: Au total, 19 articles ont été inclus. Les compétences recommandées en matière de connaissances tendent à promouvoir la compréhension des changements neurophysiologiques causés par les substances, ainsi que la connaissance de l'évaluation des comportements « à risque ¼. Les compétences couramment recommandées concernent le dépistage et la gestion des troubles liés à l'usage de substances. Les compétences recommandées en matière d'attitude comprennent l'identification des préjugés personnels et l'instauration d'une culture centrée sur le patient au sein des équipes de praticiens. Tous les articles s'appuient sur le modèle pathologique de la toxicomanie et ne reconnaissent pas les expériences potentiellement bénéfiques ou non problématiques de la consommation de substances. Pour améliorer les compétences en matière de connaissances, il est conseillé aux programmes de formation médicale d'inclure des spécialistes de la toxicomanie en tant qu'éducateurs et de prévenir la stigmatisation par le biais d'un programme d'études caché. Conclusion: Pour réduire les expériences de stigmatisation et de marginalisation parmi les patients qui consomment des substances illicites et pour améliorer la qualité des soins, les connaissances, les aptitudes et les attitudes peuvent être enseignées plus efficacement dans les programmes de formation médicale.