Preparing for Artificial General Intelligence (AGI) in Health Professions Education: AMEE Guide No. 172.

Generative Artificial Intelligence (GenAI) caught Health Professions Education (HPE) institutions off-guard, and they are currently adjusting to a changed educational environment. On the horizon, however, is Artificial General Intelligence (AGI) which promises to be an even greater leap and challenge. This Guide begins by explaining the context and nature of AGI, including its characteristics of multi-modality, generality, adaptability, autonomy, and learning ability. It then explores the implications of AGI on students (including personalised learning and electronic tutors) and HPE institutions, and considers some of the context provided by AGI in healthcare. It then raises the problems to address, including the impact on employment, social risks, student adaptability, costs, quality, and others. After considering a possible timeline, the Guide then ends by indicating some first steps that HPE institutions and educators can take to prepare for AGI.

The mediating role of self-efficacy in the relationship between past professional training and burnout resilience in medical education: a multicentre cross-sectional study.

BACKGROUND: Meta-analyses indicate a high prevalence of burnout among medical students. Although studies have investigated different coping strategies and health interventions to prevent burnout, professional experience's influence on burnout resilience as seldom been explored. Therefore, in our study we aimed to examine the self-efficacy's mediating role in the relationship between past vocational training and burnout resilience. In the process, we also analysed the associations between study-related variables and burnout resilience. METHODS: In our cross-sectional study, we analysed the data of 2217 medical students at different stages of their university education (i.e. 1st, 3rd, 6th, 10th semester, and final year) at five medical faculties in Germany. The questionnaire included items addressing variables related to medical school, previous professional and academic qualifications, and validated instruments for measuring burnout and self-efficacy. RESULTS: The overall prevalence of burnout was 19.7%, as defined by high scores for emotional exhaustion and notable values in at least one of the other two dimensions (cynicism or academic efficacy). Higher levels for self-efficacy (p < .001), having children (p = .004), and financing education with personal earnings (p = .03) were positively associated with burnout resilience, whereas having education financed by a partner or spouse (p = .04) had a negative association. In a mediation analysis, self-efficacy exerted a suppressor effect on the relationship between vocational training and burnout resilience (indirect effect = 0.11, 95% CI [0.04, 0.19]). CONCLUSIONS: Self-efficacy's suppressor effect suggests that the positive association between vocational training and burnout resilience identified in the mediation analysis disappears for students who have completed vocational training but do not feel efficacious. Those and other findings provide important insights into the psychological mechanisms underlying the development of burnout resilience in medical students and suggest the promotion of self-efficacy in medical education.

A Language Model-Powered Simulated Patient With Automated Feedback for History Taking: Prospective Study.

BACKGROUND: Although history taking is fundamental for diagnosing medical conditions, teaching and providing feedback on the skill can be challenging due to resource constraints. Virtual simulated patients and web-based chatbots have thus emerged as educational tools, with recent advancements in artificial intelligence (AI) such as large language models (LLMs) enhancing their realism and potential to provide feedback. OBJECTIVE: In our study, we aimed to evaluate the effectiveness of a Generative Pretrained Transformer (GPT) 4 model to provide structured feedback on medical students' performance in history taking with a simulated patient. METHODS: We conducted a prospective study involving medical students performing history taking with a GPT-powered chatbot. To that end, we designed a chatbot to simulate patients' responses and provide immediate feedback on the comprehensiveness of the students' history taking. Students' interactions with the chatbot were analyzed, and feedback from the chatbot was compared with feedback from a human rater. We measured interrater reliability and performed a descriptive analysis to assess the quality of feedback. RESULTS: Most of the study's participants were in their third year of medical school. A total of 1894 question-answer pairs from 106 conversations were included in our analysis. GPT-4's role-play and responses were medically plausible in more than 99% of cases. Interrater reliability between GPT-4 and the human rater showed "almost perfect" agreement (Cohen κ=0.832). Less agreement (κ<0.6) detected for 8 out of 45 feedback categories highlighted topics about which the model's assessments were overly specific or diverged from human judgement. CONCLUSIONS: The GPT model was effective in providing structured feedback on history-taking dialogs provided by medical students. Although we unraveled some limitations regarding the specificity of feedback for certain feedback categories, the overall high agreement with human raters suggests that LLMs can be a valuable tool for medical education. Our findings, thus, advocate the careful integration of AI-driven feedback mechanisms in medical training and highlight important aspects when LLMs are used in that context.

Perspectives on physician-assisted suicide in mental healthcare: results of a survey of physicians and medical students.

BACKGROUND: Physician-assisted suicide (PAS) is typically associated with serious physical illnesses that are prevalent in palliative care. However, individuals with mental illnesses may also experience such severity that life becomes intolerable. In February 2020, the previous German law prohibiting PAS was repealed. Patients with severe mental illnesses are increasingly likely to approach physicians with requests for PAS. AIMS: To explore the ethical and moral perspectives of medical students and physicians when making individual decisions regarding PAS. METHOD: An anonymised digital survey was conducted among medical students and physicians in Germany. Participants were presented with a case vignette of a chronically depressed patient requesting PAS. Participants decided on PAS provision and assessed theoretical arguments. We employed generalised ordinal regression and qualitative analysis for data interpretation. RESULTS: A total of N = 1478 participants completed the survey. Of these, n = 470 (32%) stated that they would refuse the request, whereas n = 582 (39%) would probably refuse, n = 375 (25%) would probably agree and n = 57 (4%) would definitely agree. Patient-centred arguments such as the right to self-determination increased the likelihood of consent. Concerns that PAS for chronically depressed patients might erode trust in the medical profession resulted in a decreased willingness to provide PAS. CONCLUSIONS: Participants displayed relatively low willingness to consider PAS in the case of a chronically depressed patient. This study highlights the substantial influence of theoretical medical-ethical arguments and the broader public discourse, underscoring the necessity of an ethical discussion on PAS for mental illnesses.

Assessing AI Awareness and Identifying Essential Competencies: Insights From Key Stakeholders in Integrating AI Into Medical Education.

Background: The increasing importance of artificial intelligence (AI) in health care has generated a growing need for health care professionals to possess a comprehensive understanding of AI technologies, requiring an adaptation in medical education. Objective: This paper explores stakeholder perceptions and expectations regarding AI in medicine and examines their potential impact on the medical curriculum. This study project aims to assess the AI experiences and awareness of different stakeholders and identify essential AI-related topics in medical education to define necessary competencies for students. Methods: The empirical data were collected as part of the TüKITZMed project between August 2022 and March 2023, using a semistructured qualitative interview. These interviews were administered to a diverse group of stakeholders to explore their experiences and perspectives of AI in medicine. A qualitative content analysis of the collected data was conducted using MAXQDA software. Results: Semistructured interviews were conducted with 38 participants (6 lecturers, 9 clinicians, 10 students, 6 AI experts, and 7 institutional stakeholders). The qualitative content analysis revealed 6 primary categories with a total of 24 subcategories to answer the research questions. The evaluation of the stakeholders' statements revealed several commonalities and differences regarding their understanding of AI. Crucial identified AI themes based on the main categories were as follows: possible curriculum contents, skills, and competencies; programming skills; curriculum scope; and curriculum structure. Conclusions: The analysis emphasizes integrating AI into medical curricula to ensure students' proficiency in clinical applications. Standardized AI comprehension is crucial for defining and teaching relevant content. Considering diverse perspectives in implementation is essential to comprehensively define AI in the medical context, addressing gaps and facilitating effective solutions for future AI use in medical studies. The results provide insights into potential curriculum content and structure, including aspects of AI in medicine.

Embracing digital health: German otolaryngology patients' usage and prospects of digital information and communication technologies for cross-sectoral care.

Objective: The usage of digital information and communication technologies in European healthcare is growing. Unlike numerous technological possibilities, the present use of these technologies and perspectives towards them in relation to otolaryngology care have so far been of less interest. This study evaluates the utilisation of and attitudes towards digital information and communication technologies in cross-sectoral otolaryngology care among German patients. Methods: A structured interview-based study was conducted at the outpatient facility of a tertiary hospital in Germany. It focused on chief complaints, current use of digital technologies, estimated benefits of increased digital technology use in otolaryngology care, and sociodemographic data. The detailed statistical analysis employed Chi-squared tests and multivariate logistic regression. Results: A total of 208 otolaryngology patients completed the interview. Digital communication technologies exhibited a high penetration rate (91.8%) and were regularly used in daily life (78.7%) and for health reasons (73.3%). Younger age (p ≤ 0.003) and higher education levels (p ≤ 0.008) were significantly correlated with the increased digital communication technology use. The overall potential of eHealth technologies was rated significantly higher by younger patients (p ≤ 0.001). The patients' chief complaints showed no significant influence on the current and potential use of these technologies for cross-sectoral otolaryngology care. Conclusion: Regardless of their chief complaints, German otolaryngology patients regularly use digital information and communication technologies for health reasons and express interest in their further use for cross-sectoral care. To enhance digital patient communication in otolaryngology, attention should be given to treatment quality, usability, data security and availability and financial remuneration for service providers.

Insights into discrepancies in professional identities and role models in undergraduate medical education in the context of affective burden.

Introduction: International evidence strongly suggests that medical students are at high risk of mental health problems. This distress, which can be mediated by a variety of individual, interpersonal and contextual factors within the curriculum, can be mitigated by effective coping strategies and interventions. Central to this discourse is the recognition that the challenges of professional identity formation can contribute significantly to medical students' distress. The focus of our study is therefore to examine discrepancies in professional identities and role models in undergraduate medical education in relation to affective burden. Methods: Medical students at different stages of university education and high school graduates intending to study medicine were surveyed in a cross-sectional study. The study employed Osgood and Hofstätter's polarity profile to evaluate the self-image of participants, the image of an ideal and real physician, and their correlation with depression (PHQ-9) and anxiety (GAD-7). Results: Out of the 1535 students recruited, 1169 (76.2%) participated in the study. Students rated their self-image as somewhere between a more critical real image of physicians and a more positive ideal image. Medical students at all training levels consistently rated the ideal image as remaining constant. Significant correlations were found between the professional role models of medical students and affective symptoms, particularly for the discrepancy between the ideal image of a physician and their self-image. Furthermore, 17% and nearly 15% reported significant symptoms of depression and anxiety, respectively. Discussion: Our study adds to the growing body of knowledge on professional identity formation in medicine and socialisation in the medical environment. The study highlights the importance of discrepancies between self-image and ideal image in the experience of depressive and anxiety symptoms. Primary prevention-oriented approaches should incorporate these findings to promote reflective competence in relation to professional role models and strengthen the resilience of upcoming physicians in medical training.

How professional and academic pre-qualifications relate to success in medical education: Results of a multicentre study in Germany.

OBJECTIVE: Every year, many applicants want to study medicine. Appropriate selection procedures are needed to identify suitable candidates for the demanding curriculum. Although research on medical school admissions has shown good predictive validity for cognitive selection methods (undergraduate GPA, aptitude tests), the literature on applicants with professional and/or academic experience prior to entering medical school remains slim. In our study, we therefore aimed to examine the association between academic success in medical school and having previously completed vocational training in the medical field, voluntary service (≥11 months) or an academic degree. METHODS: Data were collected in a multicentre, cross-sectional study at five medical schools in Germany (Baden-Wuerttemberg) from students during medical school (i.e. 3rd-, 6th-, and 10th-semester and final-year students). Academic success was assessed according to scores on the first and second state examinations, the total number of examinations repeated and the number of semesters beyond the standard period of study. For the analysis we calculated ordinal logistic regression models for each outcome variable of academic success. RESULTS: A total of N = 2,370 participants (response rate: RR = 47%) participated in the study. Having completed vocational training was associated with a higher amount of repeated examinations (small effect), while having an academic degree was associated with worse scores on the second state examination (medium effect). No significant association emerged between voluntary service and academic success. CONCLUSION: The results indicate that professional and academic pre-qualifications pose no advantage for academic success. Possible associations with the financing of study and living conditions of students with pre-qualifications were analysed and discussed in an exploratory manner. However, the operationalisation of academic success from objective and cognitive data should be critically discussed, as the benefits of prior experience may be captured by personal qualities rather than examination results.

The Predominance of the Health-Promoting Patterns of Work Behavior and Experience in General Practice Teams-Results of the IMPROVEjob Study.

This study aims to identify the distribution of the "Work-related behavior and experience patterns" (Arbeitsbezogenes Verhaltens-und Erlebnismuster, AVEM) in general practitioners and their teams by using baseline data of the IMPROVEjob study. Members of 60 general practices with 84 physicians in a leadership position, 28 employed physicians, and 254 practice assistants participated in a survey in 2019 and 2020. In this analysis, we focused on AVEM variables. Age, practice years, work experience, and working time were used as control variables in the Spearman Rho correlations and analysis of variance. The majority of the participants (72.1%) revealed a health-promoting pattern (G or S). Three of eleven AVEM dimensions were above the norm for the professional group "employed physicians". The AVEM dimensions "striving for perfection" (p < 0.001), "experience of success at work" (p < 0.001), "satisfaction with life" (p = 0.003), and "experience of social support" (p = 0.019) differed significantly between the groups' practice owners and practice assistants, with the practice owners achieving the higher values, except for experience of social support. Practice affiliation had no effect on almost all AVEM dimensions. We found a high prevalence of AVEM health-promoting patterns in our sample. Nearly half of the participants in all professional groups showed an unambitious pattern (S). Adapted interventions for the represented AVEM patterns are possible and should be utilized for maintaining mental health among general practice teams.

Medical students' attitudes and perceived competence regarding medical cannabis and its suggestibility.

INTRODUCTION: The global trend of legalizing medical cannabis (MC) is on the rise. In Germany, physicians have prescribed MC at the expense of health insurers since 2017. However, the teaching on MC has been scant in medical training. This study investigates medical students' attitudes and perceived competence regarding MC and evaluates how varying materials (videos/articles) impact their opinions. METHODS: Fourth-year medical students were invited to participate in the cross-sectional study. During an online session, students viewed a video featuring a patient with somatoform pain discussing her medical history, plus one of four randomly assigned MC-related materials (each an article and a video depicting a positive or negative perspective on MC). Students' opinions were measured at the beginning [T0] and the end of the course [T1] using a standardized questionnaire with a five-point Likert scale. We assessed the influence of the material on the students' opinions using paired-sample t-tests. One-way analysis of variance and Tukey post-hoc tests were conducted to compare the four groups. Pearson correlations assessed correlations. RESULTS: 150 students participated in the course, the response rate being 75.3% [T0] and 72.7% [T1]. At T0, students felt a little competent regarding MC therapy (M = 1.80 ± 0.82). At T1, students in groups 1 (positive video) and 3 (positive article) rated themselves as more capable in managing MC therapy [Formula: see text], and students in groups 3 (positive article) and 4 (negative article) felt more skilled in treating patients with chronic pain [Formula: see text]. Compared to the other groups, group 2 students (negative video) felt significantly less competent. They perceived cannabis as addictive, hazardous and unsuitable for medical prescription. DISCUSSION: This study showed that medical students lack knowledge and perceived competence in MC therapy. Material influences their opinions in different ways, and they seek more training on MC. This underlines that integrating MC education into medical curricula is crucial to address this knowledge gap.

Assessing ChatGPT's Mastery of Bloom's Taxonomy Using Psychosomatic Medicine Exam Questions: Mixed-Methods Study.

BACKGROUND: Large language models such as GPT-4 (Generative Pre-trained Transformer 4) are being increasingly used in medicine and medical education. However, these models are prone to "hallucinations" (ie, outputs that seem convincing while being factually incorrect). It is currently unknown how these errors by large language models relate to the different cognitive levels defined in Bloom's taxonomy. OBJECTIVE: This study aims to explore how GPT-4 performs in terms of Bloom's taxonomy using psychosomatic medicine exam questions. METHODS: We used a large data set of psychosomatic medicine multiple-choice questions (N=307) with real-world results derived from medical school exams. GPT-4 answered the multiple-choice questions using 2 distinct prompt versions: detailed and short. The answers were analyzed using a quantitative approach and a qualitative approach. Focusing on incorrectly answered questions, we categorized reasoning errors according to the hierarchical framework of Bloom's taxonomy. RESULTS: GPT-4's performance in answering exam questions yielded a high success rate: 93% (284/307) for the detailed prompt and 91% (278/307) for the short prompt. Questions answered correctly by GPT-4 had a statistically significant higher difficulty than questions answered incorrectly (P=.002 for the detailed prompt and P<.001 for the short prompt). Independent of the prompt, GPT-4's lowest exam performance was 78.9% (15/19), thereby always surpassing the "pass" threshold. Our qualitative analysis of incorrect answers, based on Bloom's taxonomy, showed that errors were primarily in the "remember" (29/68) and "understand" (23/68) cognitive levels; specific issues arose in recalling details, understanding conceptual relationships, and adhering to standardized guidelines. CONCLUSIONS: GPT-4 demonstrated a remarkable success rate when confronted with psychosomatic medicine multiple-choice exam questions, aligning with previous findings. When evaluated through Bloom's taxonomy, our data revealed that GPT-4 occasionally ignored specific facts (remember), provided illogical reasoning (understand), or failed to apply concepts to a new situation (apply). These errors, which were confidently presented, could be attributed to inherent model biases and the tendency to generate outputs that maximize likelihood.

A Generative Pretrained Transformer (GPT)-Powered Chatbot as a Simulated Patient to Practice History Taking: Prospective, Mixed Methods Study.

BACKGROUND: Communication is a core competency of medical professionals and of utmost importance for patient safety. Although medical curricula emphasize communication training, traditional formats, such as real or simulated patient interactions, can present psychological stress and are limited in repetition. The recent emergence of large language models (LLMs), such as generative pretrained transformer (GPT), offers an opportunity to overcome these restrictions. OBJECTIVE: The aim of this study was to explore the feasibility of a GPT-driven chatbot to practice history taking, one of the core competencies of communication. METHODS: We developed an interactive chatbot interface using GPT-3.5 and a specific prompt including a chatbot-optimized illness script and a behavioral component. Following a mixed methods approach, we invited medical students to voluntarily practice history taking. To determine whether GPT provides suitable answers as a simulated patient, the conversations were recorded and analyzed using quantitative and qualitative approaches. We analyzed the extent to which the questions and answers aligned with the provided script, as well as the medical plausibility of the answers. Finally, the students filled out the Chatbot Usability Questionnaire (CUQ). RESULTS: A total of 28 students practiced with our chatbot (mean age 23.4, SD 2.9 years). We recorded a total of 826 question-answer pairs (QAPs), with a median of 27.5 QAPs per conversation and 94.7% (n=782) pertaining to history taking. When questions were explicitly covered by the script (n=502, 60.3%), the GPT-provided answers were mostly based on explicit script information (n=471, 94.4%). For questions not covered by the script (n=195, 23.4%), the GPT answers used 56.4% (n=110) fictitious information. Regarding plausibility, 842 (97.9%) of 860 QAPs were rated as plausible. Of the 14 (2.1%) implausible answers, GPT provided answers rated as socially desirable, leaving role identity, ignoring script information, illogical reasoning, and calculation error. Despite these results, the CUQ revealed an overall positive user experience (77/100 points). CONCLUSIONS: Our data showed that LLMs, such as GPT, can provide a simulated patient experience and yield a good user experience and a majority of plausible answers. Our analysis revealed that GPT-provided answers use either explicit script information or are based on available information, which can be understood as abductive reasoning. Although rare, the GPT-based chatbot provides implausible information in some instances, with the major tendency being socially desirable instead of medically plausible information.