Assessing ChatGPT's Mastery of Bloom's Taxonomy Using Psychosomatic Medicine Exam Questions: Mixed-Methods Study.

BACKGROUND: Large language models such as GPT-4 (Generative Pre-trained Transformer 4) are being increasingly used in medicine and medical education. However, these models are prone to "hallucinations" (ie, outputs that seem convincing while being factually incorrect). It is currently unknown how these errors by large language models relate to the different cognitive levels defined in Bloom's taxonomy. OBJECTIVE: This study aims to explore how GPT-4 performs in terms of Bloom's taxonomy using psychosomatic medicine exam questions. METHODS: We used a large data set of psychosomatic medicine multiple-choice questions (N=307) with real-world results derived from medical school exams. GPT-4 answered the multiple-choice questions using 2 distinct prompt versions: detailed and short. The answers were analyzed using a quantitative approach and a qualitative approach. Focusing on incorrectly answered questions, we categorized reasoning errors according to the hierarchical framework of Bloom's taxonomy. RESULTS: GPT-4's performance in answering exam questions yielded a high success rate: 93% (284/307) for the detailed prompt and 91% (278/307) for the short prompt. Questions answered correctly by GPT-4 had a statistically significant higher difficulty than questions answered incorrectly (P=.002 for the detailed prompt and P<.001 for the short prompt). Independent of the prompt, GPT-4's lowest exam performance was 78.9% (15/19), thereby always surpassing the "pass" threshold. Our qualitative analysis of incorrect answers, based on Bloom's taxonomy, showed that errors were primarily in the "remember" (29/68) and "understand" (23/68) cognitive levels; specific issues arose in recalling details, understanding conceptual relationships, and adhering to standardized guidelines. CONCLUSIONS: GPT-4 demonstrated a remarkable success rate when confronted with psychosomatic medicine multiple-choice exam questions, aligning with previous findings. When evaluated through Bloom's taxonomy, our data revealed that GPT-4 occasionally ignored specific facts (remember), provided illogical reasoning (understand), or failed to apply concepts to a new situation (apply). These errors, which were confidently presented, could be attributed to inherent model biases and the tendency to generate outputs that maximize likelihood.

The role of face masks within in-patient psychotherapy: Results of a survey among inpatients and healthcare professionals.

Introduction: Face-to-face medical and psychotherapeutic treatments during the Corona pandemic often involve patients and health care providers wearing face masks. We performed a pilot survey assessing the subjective experience of wearing face masks during psychotherapy sessions regarding (i) feasibility, (ii) psychotherapeutic treatment and (iii) communication, emotion and working alliance in patients and healthcare professionals. Methods: A total of n = 62 inpatients (RR = 95.4%) and n = 33 healthcare professionals (RR = 86.8%) at an academic department of Psychosomatic Medicine and Psychotherapy participated in this survey anonymously. The items of the questionnaire were created by the interprofessional expert team and were based on existing instruments: (i) the Therapeutic Relationship Questionnaire and (ii) the German translation of Yalom's Questionnaire on Experiencing in Group Psychotherapy. Results: The majority of patients rate their psychotherapy as highly profitable despite the mask. In individual therapy, face masks seem to have a rather low impact on subjective experience of psychotherapy and the relationship to the psychotherapist. Most patients reported using alternative facial expressions and expressions. In the interactional group therapy, masks were rather hindering. On the healthcare professional side, there were more frequent negative associations of face masks in relation to (i) experiencing connectedness with colleagues, (ii) forming relationships, and (iii) therapeutic treatment. Discussion: Information should be given to patients about the possible effects of face masks on the recognition of emotions, possible misinterpretations and compensation possibilities through alternative stimuli (e.g., eye area) and they should be encouraged to ask for further information. Especially in group therapy, with patients from other cultural backgrounds and in cases of need for help (e.g., hearing impairment) or complex disorders, appropriate non-verbal gestures and body language should be used to match the intended emotional expression.

Where there are challenges, there are opportunities: An undergraduate medical students' teaching concept for mental health in times of COVID-19.

COVID-19 had a tremendous effect on medical education. Most teaching sessions had to be shifted online, posing additional stress and potential isolation on medical students. However, it also offered the promotion of innovative digital teaching concepts. In this article, an approach to undergraduate mental health training is presented and evaluated. The curriculum was designed according to Kern's six-step approach and consisted of asynchronous online material as well synchronous digital teaching and was accompanied by a plethora of newly developed teaching material (videos, fact sheets, etc.). Content covered the whole spectrum of diseases seen in a service of psychosomatic medicine and psychotherapy (i.e. anxiety, depression, trauma, somatoform and eating disorders, as well as motivational interviewing). Feedback from participants was collected, and exam results (written and practical) were compared to pre-COVID-19 times using t-tests for dependent and independent samples. Students were highly satisfied with the teaching (rating of 1.3 ± 0.6, n = 139 students). There was no significant difference from course evaluations before COVID-19 (1.5 ± 0.5, p > .05). The teaching also received an award in the students' competition "best digital teaching concept in summer term 2020". In the written exams, there was no significant difference between before COVID-19 (2.4 ± 0.45) and during COVID-19 times (1.6 ± 0.39; p > .05). In the practical objective structured clinical examination (OSCE), there was also no significant difference between students' judgement of the difficulty of the station (1.9 ± 0.22 vs 1.9 ± 0.31; p > .05) or how well-prepared they felt for the exam (2.0 ± 0.24 vs 2.0 ± 0.31; p > .05). However, there was a significant difference in terms of grades, with the pre-COVID-19 grades being significantly better (2.7 ± 0.37 vs 2.0 ± 0.44; p < .05), which reflects the difficulty of transferring practical skills training to an online setting. Students particularly valued the possibility of self-directed learning combined with personal guidance by departmental experts, reflecting the importance of wellbeing-centred medical education. The pandemic triggered overnight challenges for teaching mental health that may also offer the opportunity to think about worldwide teaching standards with easily accessible material and courses online. This may offer the opportunity to enthral medical students to become mental health specialists themselves.

Mental disorders are no predictors to determine the duration of cannabis-based treatment for chronic pain.

Background: Chronic pain (CP), a complex biopsychosocial disorder with a global prevalence of up to 33%, can be treated by following multidisciplinary approaches that may include cannabis-based medicine (CBM). However, because CBM continues to be a new treatment, questions remain regarding the ideal duration for CBM and its psychosocial determinants, including mental comorbidities. Methods: In a retrospective cross-sectional study involving 46 patients with CP (ICD-10 code F45.4-), three validated instruments-the German Pain Questionnaire, the Depression Anxiety Stress Scale (DASS), and the Marburg Questionnaire of Habitual WellBeing-were used to identify pain-specific psychosocial determinants and mental disorders. Descriptive analyses, a group differences analysis, and a logistic regression analysis were performed using SPSS. Results: The patients most frequently reported low back pain as the primary location of their CP, and in attributing the condition to tissue damage, most had largely adopted a somatic orientation in conceptualizing their illness. Most had experienced CP for more than 5 years (M = 5.13 years, SD = 1.41) and, as a consequence, faced significant restrictions in their everyday life and exhibited low subjective wellbeing (MFHW median = 4.00, N = 43, Q1: 2.00, Q3: 9.00, range: 0-20). Comorbidities among the patients included depression, (DASS-Depression, median: 11.50, Q1: 7.00, Q3: 16.25), anxiety (DASS-Anxiety, median: 4.50, Q1: 2.75, Q3: 8.00), and stress (DASS-Stress, median: 11.00, Q1: 7.00, Q3: 15.00). Between the two cannabis-based treatments with a course lasting either less or more than a year, the duration of treatment showed no between-group differences in terms of sociodemographic factors, pain-specific factors, conceptualizations of the illness, or mental disorders. Psychosocial determinants such as subjective wellbeing and mental comorbidities were not significant predictors of the duration of cannabis-based treatment. Conclusion: We found no evidence indicating that the benefits of short-term vs. long-term cannabis-based treatment can be predicted by mental comorbidities or psychosocial factors. However, because CBM may be included in approaches to treat CP, questions about the ideal duration of such treatment remain to be answered.