Matching of advanced undergraduate medical students' competence profiles with the required competence profiles of their specialty of choice for postgraduate training.

BACKGROUND: Matching between undergraduate students and their chosen specialty has implications for their personal job satisfaction and performance as well as societies' needs regarding health care quality. Knowledge regarding student-specialty fit can help improve students' decisions and detect potential deficiencies in specific competences. In this study, we compare self-assessed competence profiles of medical students close to graduation with the competence profiles of their specialty of choice for postgraduate training. METHODS: Self-assessed competence profiles were collected with the modified requirement-tracking (R-Track) questionnaire from 197 final-year medical students close to graduation in 2022. To determine student-specialty fit, difference scores between students' self-assessed competences and physicians' requirements for specific specialties were calculated across the R-Track's six competence areas "Motivation", "Personality traits", "Social interactive competences", "Mental abilities", "Psychomotor & multitasking abilities", and "Sensory abilities", which were assessed on a 5-point Likert scale (1: "very low" to 5: "very high"). Mean difference scores across competence areas were calculated and compared between specialties with multivariate analysis of variance. Student-specialty fit was also calculated independent of students' choices. RESULTS: The competence area "Motivation" scored highest for both students and physicians across specialties. However, students' scores were lower than physicians' requirements for "Motivation" as well as "Personality traits" across all specialties. Difference scores for "Social interactive competences" were either close to zero or showed higher scores for students. A similar competence pattern for internal medicine, general medicine, paediatrics, and gynaecology was identified with higher than required student scores for "Mental abilities", "Psychomotor & multitasking abilities", and "Sensory abilities". All other specialties showed higher physicians' requirements for at least one of these competence areas. Independent of students' specialty choice, we found the highest difference score in favour of student scores for general medicine (0.31) and the lowest difference score for internal medicine (-0.02). CONCLUSIONS: Students' competence profiles overall show better fit with person-oriented specialties. "Mental abilities", "Psychomotor & multitasking abilities", and "Sensory abilities" show higher requirement scores for more technique-oriented specialties. Students interested in such specialties could focus more on basic skill development in undergraduate training or will develop specific skills during residency.

How does multisource feedback influence residency training? A qualitative case study.

INTRODUCTION: Multisource feedback (MSF), also called 360-degree assessment, is one form of assessment used in postgraduate training. However, there is an ongoing discussion on its value, because the factors that influence the impact of MSF and the main impact of MSF are not fully understood. In this study, we investigated both the influencing factors and the impact of MSF on residency training. METHODS: We conducted a qualitative case study within the boundaries of the residency training for paediatricians and paediatric surgeons at a University Hospital. We collected data from seven focus group interviews with stakeholders of MSF (residents, raters and supervisors). By performing a reflexive thematic analysis, we extracted the influencing factors and the impact of MSF. RESULTS: We found seven influencing factors: MSF is facilitated by the announcement of a clear goal of MSF, the training of raters on the MSF instrument, a longitudinal approach of observation, timing not too early and not too late during the rotation, narrative comments as part of the ratings, the residents' self-assessment and a supervisor from the same department. We found three themes on the impact of MSF: MSF supports the professional development of residents, enhances interprofessional teamwork and increases the raters' commitment to the training of residents. CONCLUSION: This study illuminates the influencing factors and impact of MSF on residency training. We offer novel recommendations on the continuity of observation, the timing during rotations and the role of the supervisor. Moreover, by discussing our results through the lens of identity formation theory, this work advances our conceptual understanding of MSF. We propose identity formation theory as a framework for future research on MSF to leverage the potential of MSF in residency training.

Effective methods to enhance medical students' cardioversion and transcutaneous cardiac pacing skills retention - a prospective controlled study.

BACKGROUND: Guideline-based therapy of cardiac arrhythmias is important for many physicians from the beginning of their training. Practical training of the required skills to treat cardiac arrhythmias is useful for acquiring these skills but does not seem sufficient for skill retention. The aim of this study was to compare different retention methods for skills required to treat cardiac arrhythmias with respect to the performance of these skills in an assessment. METHODS: Seventy-one final-year medical students participated in a newly designed workshop to train synchronized cardioversion (SC) and transcutaneous cardiac pacing (TCP) skills in 2020. All participants completed an objective structured clinical examination (OSCE 1) one week after the training. Afterwards, the participants were stratified and randomized into three groups. Nine weeks later, one group received a standard operating procedure (SOP) for the skills, one group participated in a second workshop (SW), and one group received no further intervention (control). Ten weeks after the first training, all groups participated in OSCE 2. RESULTS: The average score of all students in OSCE 1 was 15.6 ± 0.8 points with no significant differences between the three groups. Students in the control group reached a significantly (p < 0.001) lower score in OSCE 2 (-2.0 points, CI: [-2.9;-1.1]) than in OSCE 1. Students in the SOP-group achieved on average the same result in OSCE 2 as in OSCE 1 (0 points, CI: [-0.63;+0.63]). Students who completed a second skills training (SW-group) scored not significantly higher in OSCE 2 compared to OSCE 1 (+0.4 points, CI: [-0.29;+1.12]). The OSCE 2 scores in groups SOP and SW were neither significantly different nor statistically equivalent. CONCLUSIONS: Partial loss of SC and TCP skills acquired in a workshop can be prevented after 10 weeks by reading an SOP as well as by a second workshop one week before the second assessment. Refreshing practical skills with an SOP could provide an effective and inexpensive method for skills retention compared to repeating a training. Further studies need to show whether this effect also exists for other skills and how frequently an SOP should be re-read for appropriate long-term retention of complex skills.

Advanced undergraduate medical students' perceptions of basic medical competences and specific competences for different medical specialties - a qualitative study.

BACKGROUND: Medical graduates should have acquired basic competences that enable them to practice medicine independently as physicians and to enter postgraduate training in any specialty they wish. Little is known about advanced undergraduate medical students' perceptions of basic medical competences needed to start postgraduate training and about specialty-specific competences. This qualitative study aims to identify medical students' perceptions of basic medical competences and specific competence requirements for different specialties. METHODS: In December 2020, sixty-four advanced undergraduate medical students participated in the role of a resident in a competence-based telemedicine training simulating a first day in postgraduate training. After the training, eight focus group interviews were conducted about students' perceptions of basic medical competences and specialty-specific competences using a semi-structured interview guide. The interviews were transcribed and analysed thematically according to the six steps of Braun and Clarke. The analysis was carried out by an inductive search for themes, which were deductively assigned to the six competence areas of the requirement-tracking questionnaire (R-Track). RESULTS: Regarding basic medical competences, four R-Track competence areas could be identified as main themes. The students considered 'Social-interactive competences' to be particularly relevant for basic clinical work, including 'Structuring information', 'Tactfulness', and 'Stress resistance'. Students especially emphasized 'Concentration' as an important aspect of the competence area 'Mental abilities'. Among 'Personality traits', 'Honesty' was mentioned most frequently, and students were also aware that 'Expertise' is particularly important for 'Motivation'. For different specialties, some competence areas were newly added to the competences needed for the respective specialty. For surgery, the competence areas 'Sensory abilities' and 'Psychomotor & multitasking abilities' were mentioned anew. 'Sensory abilities' were also newly attributed to radiology. 'Mental abilities' were mentioned as new competence area for psychiatry and internal medicine, while for anaesthesiology, 'Psychomotor & multitasking abilities' were newly added. CONCLUSIONS: Advanced students seem to be well aware of basic competences needed for clinical practice. Good consensus between students and physicians was only found for psychiatry-specific competences. Medical schools should support their students in matching their perceptions of competences needed for specific specialties with specialty-specific requirements for a realistic choice of a specialty for postgraduate training.

Final-year medical students' self-assessment of facets of competence for beginning residents.

BACKGROUND: Final-year undergraduate medical students often do not feel well prepared for their start of residency training. Self-assessment of competences is important so that medical trainees can take responsibility for their learning. In this study, we investigated how final-year medical students self-assessed their competences as they neared their transition to postgraduate training. The aim was to identify areas for improvement in undergraduate training. METHODS: In the academic year 2019/2020, a national online survey was sent to final-year undergraduate medical students via their respective medical schools. The survey included ten facets of competence (FOC) most relevant for beginning residents. The participants were asked to self-assess their competence for each FOC on a 5-point Likert scale (1: strongly disagree to 5: strongly agree). We established an order of self-assessed FOC performance by means and calculated paired t-tests. Gender differences were assessed with independent t-tests. RESULTS: A total of 1083 students from 35 medical schools completed the questionnaire. Mean age was 27.2 ± 3.1 years and 65.8% were female. Students rated their performance highest in the FOCs 'Teamwork and collegiality' and 'Empathy and openness' (97.1 and 95.0% 'strongly agree' or 'agree', respectively) and lowest in 'Verbal communication with colleagues and supervisors' and 'Scientifically and empirically grounded method of working' (22.8 and 40.2% 'strongly disagree', 'disagree', or 'neither agree nor disagree', respectively). Women rated their performance of 'Teamwork and collegiality', 'Empathy and openness', and 'Knowing and maintaining own personal bounds and possibilities' significantly higher than men did (Cohen's d > .2), while men showed higher self-assessed performance in 'Scientifically and empirically grounded method of working' than women (Cohen's d = .38). The FOCs 'Responsibility', 'Knowing and maintaining own personal bounds and possibilities', 'Structure, work planning, and priorities', 'Coping with mistakes', and 'Scientifically and empirically grounded method of working' revealed lower self-assessed performance than the order of FOC relevance established by physicians for beginning residents. CONCLUSIONS: The differences between the level of students' self-assessed FOC performance and physicians' ranking of FOC relevance revealed areas for improvement in undergraduate medical education related to health system sciences. Final-year students might benefit from additional or better training in management skills, professionalism, and evidence-based medicine. Surveys of self-assessed competences may be useful to monitor competence development during undergraduate training.

Membranous nephropathy and primary biliary cholangitis: A case report and review of the literature.

AIM: Membranous nephropathy (MN) and primary biliary cholangitis (PBC) are autoimmune diseases that coexist in some cases and might share a common pathogenesis. In 75 - 80% of MN patients, PLA2R1 or THSD7A is the target antigen responsible for disease development, while in the remaining cases, MN pathogenesis is not clear. Our aim was to identify potential antigens playing an overlapping pathogenic role for development of both PBC and MN. MATERIALS AND METHODS: Serum from a patient with PBC-associated MN was analyzed for MN and PBC-specific autoantibodies and kidney biopsy tissue was stained for the respective antigens. A review of the literature for published PBC-associated MN cases was performed. RESULTS: A 39-year-old male patient was diagnosed with PBC-associated MN. Serology tests revealed negativity for PLA2R1-ab and THSD7A-ab, but positivity for two PBC-specific antibodies: M2-ab and gp210-ab. Kidney biopsy was stained for both PBC-specific antigens, PDC-E2 and gp210, as well as PLA2R1 and THSD7A, showing no MN-specific positivity. Human glomerular extracts also did not contain PDC-E2 or gp210. A review of all 17 published cases of PBC-associated MN showed that 71% of patients suffered from at least one additional autoimmune disease, and different IgG-subclasses were found in the renal immune deposits of these patients. CONCLUSION: These results indicate that both PBC-antigens are not the putative antigen(s) leading to MN development in this patient. PBC-antigens might not be directly responsible for MN development. Both diseases seem to present as autoimmune phenomena triggered by interaction between unknown factors.

It does not have to be either or! Assessing competence in medicine should be a continuum between an analytic and a holistic approach.

Assessing competence is a tremendous challenge in medical education. There are two contrasting approaches in competence assessment: an analytic approach that aims to precisely measure observable constituents and facets of competence and a holistic approach that focuses on a comprehensive assessment of competences in complex real situations reflecting actual performance. We would like to contribute to the existing discourse about medical competence and its assessment by proposing an approach that can provide orientation for the development of competence-based assessment concepts in undergraduate and postgraduate medical education. The approach follows Kane's framework of an "argument-based approach" to validity and is based on insights into task complexity, testing and learning theories as well as the importance of the learning environment. It describes a continuum from analytic to holistic approaches to assess the constituents and facets of competence to performance. We conclude that the complexity of a task should determine the selection of the assessment and suggest to use this approach to reorganize and adapt competence assessment.

Are different medical school admission tests associated with the outcomes of a simulation-based OSCE?

BACKGROUND: Medical school admission procedures have the common goal to select applicants with the greatest potential of becoming successful physicians. Hamburg Medical Faculty selects medical students by grade point average (GPA) and employs a two-step selection process of a natural sciences test (HAM-Nat), in some cases followed by multiple mini-interviews (HAM-Int). Multiple mini-interviews can predict non-cognitive outcomes, while GPA has predictive validity for cognitive outcomes. The aim of our study was to explore communication skills and clinical knowledge of advanced medical students according to their respective admission procedure. METHODS: In July 2019, 146 students grouped according to their admission procedure into GPA-only (19.2 %), HAM-Nat (33.6 %), HAM-Int (30.8 %), and Waiting List (16.4 %) participated in four OSCE stations which equally assessed students' communication skills (OSCE part 1) and clinical knowledge (OSCE part 2) in simulated patient encounters, rated by physicians with checklists. Additionally, psychosocial assessors ranked communication skills with a global rating scale (GR). The students also participated in a multiple choice (MC) exam testing clinical knowledge. Kruskal-Wallis analyses of variance of test performance and Spearman correlation of instruments were calculated. RESULTS: Students from the Waiting List group performed significantly worse on the MC exam compared to GPA-only and HAM-Int (adjusted p = .029 and 0.018, respectively). No significant differences were found between the admission groups with respect to communication skills. Global Rating and OSCE part 1 (communication) correlated significantly (ρ = 0.228, p = .006) as did OSCE part 2 (clinical knowledge) and MC exam (ρ = 0.242, p = .003), indicating criterion validity. Constructs did not overlap, indicating divergent validity. CONCLUSIONS: Advanced medical students selected for undergraduate studies by multiple mini-interviews assessing psychosocial skills showed similar communication skills compared to students admitted to medical school by other entryways. It is unclear whether these similarities are due to an effective undergraduate longitudinal communication curriculum. Assessing baseline communication skills of all medical students at entry-level may aid with this question.

Defining competence profiles of different medical specialties with the requirement-tracking questionnaire - a pilot study to provide a framework for medial students' choice of postgraduate training.

BACKGROUND: The medical specialties are characterised by a great diversity in their daily work which requires different sets of competences. A requirement analysis would help to establish competence profiles of the different medical specialities. The aim of this pilot study was to define competence profiles for individual medical specialties. This could provide a framework as support for medical graduates who wish to choose a medical specialty for their postgraduate training. METHODS: In February 2020, physicians were invited via the State Chamber of Physicians' monthly journal to electronically fill out the requirement tracking (R-Track) questionnaire. It contains 63 aspects assigned to six areas of competence: "Mental abilities", "Sensory abilities", "Psychomotor and multitasking abilities", "Social interactive competences", "Motivation", and "Personality traits". The expression of the different aspects was assessed on a 5-point Likert scale (1: "very low" to 5: "very high"). Sociodemographic data and information about the current workplace (hospital or practice) were also collected. RESULTS: In total, 195 practicing physicians from 19 different specialities followed the invitation by the State Chamber of Physicians to participate in this survey. For almost all medical specialties, the competence area "Motivation" reached rank 1. "Psychomotor and multitasking abilities" received high ranks among specialties performing surgical activities, while "Social interactive competences" and "Personality traits" were highly rated by specialties with an intense level of patient-physician-interaction. "Mental abilities" were only rated highly by radiologists (rank 2) and physiologists (rank 3) while "Sensory abilities" were generally rated very low with the expression (rank 4) for anaesthesiology and ENT. CONCLUSIONS: In this pilot study, a first outline of competences profiles for 17 medical specialties were defined. The specific "Motivation" for a medical specialty seemed to play the greatest role for most specialties. This first specialty specific competence framework could provide a first insight into specific competences required by medical specialties and could serve medical graduate as a decision aid when looking for a medical specialty for their postgraduate training.

Final-year medical students' competence profiles according to the modified requirement tracking questionnaire.

BACKGROUND: Undergraduate medical education is supposed to equip medical students with basic competences to select any specialty of their choice for postgraduate training. Medical specialties are characterized by a great diversity of their daily work routines and require different sets of competence facets. This study examines the self-assessed competence profiles of final-year undergraduate medical students and their specialty choice for postgraduate training. Students' profiles, who wish to choose anaesthesiology, internal medicine, or paediatrics, are compared with the physicians' competence profiles from these three disciplines. METHODS: In this study, 148 volunteer final-year undergraduate medical students completed the modified requirement-tracking (R-Track) questionnaire for self-assessment of their competence profiles. The R-Track questionnaire contains 63 competence facets assigned to six areas of competence: "Mental abilities", "Sensory abilities", "Psychomotor & multitasking abilities", "Social interactive competences", "Motivation", and "Personality traits". The expression of the different competence facets had to be assessed on a 5-point Likert scale (1: "very low" to 5: "very high"). Additionally, socio-demographic data and the participants' first choice of a medical speciality for postgraduate education were collected. We used analysis of variance (ANOVA) for mean score comparison of subgroups and least significant difference (LSD) tests for post hoc analysis. RESULTS: The competence area with the highest rating was "Motivation" (3.70 ± 0.47) while "Psychomotor & multitasking abilities" received the lowest rating (3.34 ± 0.68). Individual facets of competence ranked from "In need of harmony" (4.36 ± 0.72), followed by "Tactfulness" (4.26 ± 0.64), and "Cooperation/Agreeableness" (4.24 ± 0.53) to "Risk orientation" (2.90 ± 0.92), "Mathematical reasoning" (2.87 ± 1.25), and "Sanctioning" (2.26 ± 0.93). The students' competence profiles showed 100 % congruence with physicians' competence profiles of the postgraduate specialty of their choice for internal medicine, 33.3 % for paediatrics, and 0 % for anaesthesiology. CONCLUSIONS: Undergraduate medical students could define their competence profiles with the modified R-Track questionnaire and compare them with the profile of their desired specialty for postgraduate training to discover possible learning gaps or to detect good specialty matches. A combination of students' competence self-assessment with an external assessment of students' facets of competence could provide curricular planners with useful information how to design learning opportunities for specific facets of competence.

Entrustable professional activities and facets of competence in a simulated workplace-based assessment for advanced medical students.

Background: Competence-based assessment formats in medical education usually focus on individual facets of competence (FOCs). The concept of 'Entrustable Professional Activities' (EPAs) encompasses supervisors' decisions on which level of supervision a trainee requires to perform a professional activity including several FOCs. How the different FOCs as perceived by clinician raters contribute to entrustment decisions is yet unclear.Objective: How do FOC perceptions relate to entrustment-decisions?Methods: Sixty-seven advanced medical students participated in an assessment simulating the first day of a resident physician. Participants were rated by supervisors for seven FOCs and twelve EPAs.Results: There was a positive correlation between FOC and EPA scores. Each EPA displayed a different correlation pattern with FOC ratings.Discussion: For most EPAs high levels of entrustment were associated with high ratings for selected FOCs. The results are in alignment with the assumption that each EPA encompasses a different set of FOCs.Conclusions: In our simulated workplace-based assessment, entrustment decisions for EPAs reflect the FOCs observed in a trainee. Thus, assessment of FOCs alongside with EPA ratings could add to the understanding of factors contributing to entrustment decisions.

Need for cognitive closure, tolerance for ambiguity, and perfectionism in medical school applicants.

BACKGROUND: Physicians have to deal with uncertainty on a daily basis, which requires high tolerance for ambiguity. When medical decisions have to be made in ambiguous situations, low levels of need for cognitive closure and high levels of adaptive perfectionism are beneficial. It might be useful to measure such personality traits during medical school selection processes. In our study, we explored the expression of need for cognitive closure, tolerance for ambiguity, and perfectionism in medical school applicants who participated in a multiple mini-interview selection process with respect to the final decision of admission or rejection. METHODS: After participating in the multiple mini-interview procedure (HAM-Int) at Hamburg Medical School in August 2019, 189 medical school applicants filled out a questionnaire including the Multidimensional Perfectionism Scale by Hewitt and Flett (MPS-H), the Multidimensional Perfectionism Scale by Frost (MPS-F), the Tolerance for Ambiguity Scale (TAS), the 16-Need for Cognitive Closure Scale (16-NCCS), and sociodemographic data. After the final admission decision, the scores of need for cognitive closure, tolerance for ambiguity, and perfectionism of admitted and rejected applicants were compared. We also assessed the predictive power of need for cognitive closure and age for the admission decision in a binary logistic regression model. RESULTS: Compared to the admitted applicants, the rejected applicants showed a significantly higher need for cognitive closure (p = .009). A high need for cognitive closure correlated significantly positively with maladaptive perfectionism (p < .001) and significantly negatively with tolerance for ambiguity (p < .001). Low need for cognitive closure and older age were associated with a positive admission decision. CONCLUSIONS: Regarding the personality traits need for cognitive closure, tolerance for ambiguity, and perfectionism we identified interesting differences and correlations of relevance for physicians' daily work in medical school applicants who were admitted or rejected after participating in a multiple mini-interview selection procedure. Further studies are needed to investigate these characteristics and their development longitudinally in medical students and to correlate them with students' medical performance.

Implicit expression of uncertainty - suggestion of an empirically derived framework.

BACKGROUND: Uncertainty occurs in physicians' daily work in almost every clinical context and is also present in the clinical reasoning process. The way physicians communicate uncertainty in their thinking process during handoffs is crucial for patient safety because uncertainty has diverse effects on individuals involved in patient care. Dealing with uncertainty and expressing uncertainty are important processes in the development of professional identity of undergraduate medical students. Many studies focused on how to deal with uncertainty and whether uncertainty is explicitly expressed. Hardly any research has been done regarding implicit expression of uncertainty. Therefore, we studied the ways in which medical students in the role of beginning residents implicitly express uncertainty during simulated handoffs. METHODS: Sixty-seven advanced undergraduate medical students participated in a simulated first day of residency including a consultation hour, a patient management phase with interprofessional interaction, and a patient handoff. We transcribed the videographed handoffs verbatim and extracted language with respect to expression of uncertainty using a grounded theory approach. Text sequences expressing patient related information were analyzed and coded with respect to language aspects which implicitly modified plain information with respect to increasing or decreasing uncertainty. Concepts and categories were developed and discussed until saturation of all aspects was reached. RESULTS: We discovered a framework of implicit expressions of uncertainty regarding diagnostic and treatment-related decisions within four categories: "Statement", "Assessment", "Consideration", and "Implication". Each category was related to either the subcategory "Actions" or "Results" within the diagnostic or therapeutic decisions. Within each category and subcategory, we found a subset of expressions, which implicitly attenuated or strengthened plain information thereby increasing uncertainty or certainty, respectively. Language that implicitly attenuated plain information belonged to the categories questionable, incomplete, alterable, and unreliable while we could ascribe implicit strengtheners to the categories assertive, adequate, focused, and reliable. CONCLUSIONS: Our suggested framework of implicit expression of uncertainty may help to raise the awareness for expression of uncertainty in the clinical reasoning process and provide support for making uncertainty explicit in the teaching process. This may lead to more transparent communication processes among health care professionals and eventually to improved patient safety.

Assessing clinical reasoning in undergraduate medical students during history taking with an empirically derived scale for clinical reasoning indicators.

BACKGROUND: The clinical reasoning process, which requires biomedical knowledge, knowledge about problem-solving strategies, and knowledge about reasons for diagnostic procedures, is a key element of physicians' daily practice but difficult to assess. The aim of this study was to empirically develop a Clinical Reasoning Indicators-History Taking-Scale (CRI-HT-S) and to assess the clinical reasoning ability of advanced medical students during a simulation involving history taking. METHODS: The Clinical Reasoning Indictors-History Taking-Scale (CRI-HT-S) including a 5-point Likert scale for assessment was designed from clinical reasoning indicators identified in a qualitative study in 2017. To assess indicators of clinical reasoning ability, 65 advanced medical students (semester 10, n = 25 versus final year, n = 40) from three medical schools participated in a 360-degree competence assessment in the role of beginning residents during a simulated first workday in hospital. This assessment included a consultation hour with five simulated patients which was videotaped. Videos of 325 patient consultations were assessed using the CRI-HT-S. A factor analysis was conducted and the students' results were compared according to their advancement in undergraduate medical training. RESULTS: The clinical reasoning indicators of the CRI-HT-S loaded on three factors relevant for clinical reasoning: 1) focusing questions, 2) creating context, and 3) securing information. Students reached significantly different scores (p < .001) for the three factors (factor 1: 4.07 ± .47, factor 2: 3.72 ± .43, factor 3: 2.79 ± .83). Students in semester 10 reached significantly lower scores for factor 3 than students in their final year (p < .05). CONCLUSIONS: The newly developed CRI-HT-S worked well for quantitative assessment of clinical reasoning indicators during history taking. Its three-factored structure helped to explore different aspects of clinical reasoning. Whether the CRI-HT-S has the potential to be used as a scale in objective structured clinical examinations (OCSEs) or in workplace-based assessments of clinical reasoning has to be investigated in further studies with larger student cohorts.

A german-language competency-based multisource feedback instrument for residents: development and validity evidence.

BACKGROUND: In medical settings, multisource feedback (MSF) is a recognised method of formative assessment. It collects feedback on a doctor's performance from several perspectives in the form of questionnaires. Yet, no validated MSF questionnaire has been publicly available in German. Thus, we aimed to develop a German MSF questionnaire based on the CanMEDS roles and to investigate the evidence of its validity. METHODS: We developed a competency-based MSF questionnaire in German, informed by the literature and expert input. Four sources of validity evidence were investigated: (i) Content was examined based on MSF literature, blueprints of competency, and expert-team discussions. (ii) The response process was supported by analysis of a think-aloud study, narrative comments, "unable to comment" ratings and evaluation data. (iii) The internal structure was assessed by exploratory factor analysis, and inter-rater reliability by generalisability analysis. Data were collected during two runs of MSF, in which 47 residents were evaluated once (first run) or several times (second and third run) on 81 occasions of MSF. (iv) To investigate consequences, we analysed the residents' learning goals and the progress as reported via MSF. RESULTS: Our resulting MSF questionnaire (MSF-RG) consists of 15 items and one global rating, which are each rated on a scale and accompanied by a field for narrative comments and cover a construct of a physician's competence. Additionally, there are five open questions for further suggestions. Investigation of validity evidence revealed that: (i) The expert group agreed that the content comprehensively addresses clinical competence; (ii) The response processes indicated that the questions are understood as intended and supported the acceptance and usability; (iii) For the second run, factor analysis showed a one-factor solution, a Cronbach's alpha of 0.951 and an inter-rater reliability of 0.797 with 12 raters; (iv) There are indications that residents benefitted, considering their individual learning goals and based on their ratings reported via MSF itself. CONCLUSIONS: To support residency training with multisource feedback, we developed a German MSF questionnaire (MSF-RG), which is supported by four sources of validity evidence. This MSF questionnaire may be useful to implement MSF in residency training in German-speaking regions.

Validation of a competence-based assessment of medical students' performance in the physician's role.

BACKGROUND: Assessing competence of advanced undergraduate medical students based on performance in the clinical context is the ultimate, yet challenging goal for medical educators to provide constructive alignment between undergraduate medical training and professional work of physicians. Therefore, we designed and validated a performance-based 360-degree assessment for competences of advanced undergraduate medical students. METHODS: This study was conducted in three steps: 1) Ten facets of competence considered to be most important for beginning residents were determined by a ranking study with 102 internists and 100 surgeons. 2) Based on these facets of competence we developed a 360-degree assessment simulating a first day of residency. Advanced undergraduate medical students (year 5 and 6) participated in the physician's role. Additionally knowledge was assessed by a multiple-choice test. The assessment was performed twice (t1 and t2) and included three phases: a consultation hour, a patient management phase, and a patient handover. Sixty-seven (t1) and eighty-nine (t2) undergraduate medical students participated. 3) The participants completed the Group Assessment of Performance (GAP)-test for flight school applicants to assess medical students' facets of competence in a non-medical context for validation purposes. We aimed to provide a validity argument for our newly designed assessment based on Messick's six aspects of validation: (1) content validity, (2) substantive/cognitive validity, (3) structural validity, (4) generalizability, (5) external validity, and (6) consequential validity. RESULTS: Our assessment proved to be well operationalised to enable undergraduate medical students to show their competences in performance on the higher levels of Bloom's taxonomy. Its generalisability was underscored by its authenticity in respect of workplace reality and its underlying facets of competence relevant for beginning residents. The moderate concordance with facets of competence of the validated GAP-test provides arguments of convergent validity for our assessment. Since five aspects of Messick's validation approach could be defended, our competence-based 360-degree assessment format shows good arguments for its validity. CONCLUSION: According to these validation arguments, our assessment instrument seems to be a good option to assess competence in advanced undergraduate medical students in a summative or formative way. Developments towards assessment of postgraduate medical trainees should be explored.

The relationship between perfectionism and symptoms of depression in medical school applicants.

BACKGROUND: The association between perfectionism and depression in the medical profession can ultimately influence physicians' performance negatively. In medical students, especially maladaptive perfectionism is connected with distress and lower academic performance. The expression of perfectionism and symptoms of depression at the time of medical school application is not known. Therefore, we explored perfectionism and symptoms of depression in participants of multiple mini-interviews for medical school admission and investigated possible differences between applicants who were eventually admitted or rejected. METHODS: After the multiple mini-interviews admission procedure at Hamburg Medical School in August 2018, 146 applicants filled out a questionnaire including sociodemographic data and the following validated instruments: Multidimensional Perfectionism Scale by Hewitt and Flett (MPS-H), Multidimensional Perfectionism Scale by Frost (MPS-F), Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder Scale (GAD-7), and a 10-item version of the Big Five Inventory (BFI-10). The two groups of admitted and rejected applicants were compared and the correlation between symptoms of depression and perfectionism further explored. RESULTS: The admitted applicants were significantly more extrovert and had lower depression scores compared to the rejected applicants. In both groups, the composite scales of Adaptive Perfectionism (r = .21, p = .011) and Maladaptive Perfectionism (r = .43, p < .001) as well as their components correlated significantly with the PHQ-9 results. Maladaptive Perfectionism accounted for about 18% of variance in the PHQ-9 score. CONCLUSIONS: Rejected medical school applicants who participated in a multiple mini-interviews admission procedure showed higher levels of depression symptoms than admitted applicants. The degree of depressive symptoms can be partly explained by Maladaptive Perfectionism scores. Since coping in medical school and in postgraduate medical education require robust mental health, perfectionism questionnaires could be an additional tool in medical school selection processes.

"Princess and the pea" - an assessment tool for palpation skills in postgraduate education.

BACKGROUND: In osteopathic medicine, palpation is considered to be the key skill to be acquired during training. Whether palpation skills are adequately acquired during undergraduate or postgraduate training is difficult to assess. The aim of our study was to test a palpation assessment tool developed for undergraduate medical education in a postgraduate medical education (PME) setting. METHODS: We modified and standardized an assessment tool, where a coin has to be palpated under different layers of copy paper. For every layer depth we randomized the hiding positions with a random generator. The task was to palpate the coin or to determine that no coin was hidden in the stack. We recruited three groups of participants: 22 physicians with no training in osteopathic medicine, 25 participants in a PME course of osteopathic techniques before and after a palpation training program, 31 physicians from an osteopathic expert group with at least 700 h of osteopathic skills training. These experts ran the test twice to check for test-retest-reliability. Inferential statistical analyzes were performed using generalized linear mixed models with the dichotomous variable "coin detected / not detected" as the dependent variable. RESULTS: We measured a test-retest reliability of the assessment tool as a whole with 56 stations in the expert group of 0.67 (p <  0.001). For different paper layers, we found good retest reliabilities up to 300 sheets. The control group detected a coin significantly better in a depth of 150 sheets (p = 0.01) than the pre-training group. The osteopathic training group showed significantly more correct coin localizations after the training in layer depths of 200 (p = 0.03) and 300 sheets (p = 0.05). This group also had significantly better palpation results than the expert group in the depth of 300 sheets (p = 0.001). When there was no coin hidden, the expert group showed significantly better results than the post-training group (p = 0.01). CONCLUSIONS: Our tool can be used with reliable results to test palpation course achievements with 200 and 300 sheets of paper. Further refinements of this tool will be needed to use it in complex assessment designs for the evaluation of more sophisticated palpatory skills in postgraduate medical settings.

Interprofessional assessment of medical students' competences with an instrument suitable for physicians and nurses.

BACKGROUND: Physicians need a set of specific competences to perform well in interprofessional teams in their first year of residency. These competences should be achieved with graduation from medical school. Assessments during undergraduate medical studies are mostly rated by supervisors only. The aim of our study was to compare the rating of core facets of competence of medical students late in their undergraduate training as well as the rating confidence between three different groups of assessors (supervisors, residents, and nurses) in an assessment simulating the first day of residency. METHODS: Sixty-seven advanced medical students from three different medical schools (Hamburg, Oldenburg and Munich) participated in a 360-degree assessment simulating the first working day of a resident. Each participant was rated by three assessors - a supervisor, a resident and a nurse - in seven facets of competence relevant for the first year of residency: (1) responsibility, (2) teamwork and collegiality, (3) knowing and maintaining own personal bounds and possibilities, (4) structure, work planning and priorities, (5) coping with mistakes, (6) scientifically and empirically grounded method of working, and (7) verbal communication with colleagues and supervisors. Means of all assessed competences and confidences of judgement of the three rating groups were compared. Additionally, correlations between assessed competences and confidence of judgement within each group of raters were computed. RESULTS: All rating groups showed consistent assessment decisions (Cronbach's α: supervisors = .90, residents = .80, nurses = .78). Nurses assessed the participants significantly higher in all competences compared to supervisors and residents (all p ≤ .05) with moderate and high effect sizes (d = .667-1.068). While supervisors' and residents' ratings were highest for "teamwork and collegiality", participants received the highest rating by nurses for "responsibility". Competences assessed by nurses were strongly positively correlated with their confidence of judgment while supervisors' assessments correlated only moderately with their confidence of judgment in two competences. CONCLUSIONS: Different professional perspectives provide differentiated competence ratings for medical students in the role of a beginning resident. Rating confidence should be enhanced by empirically derived behavior checklists with anchors, which need to be included in rater training to decrease raters' subjectivity.

Assessing core competences of medical students with a test for flight school applicants.

BACKGROUND: Important competences of physicians regarding patient safety include communication, leadership, stress resistance, adherence to procedures, awareness, and teamwork. Similarly, while selected, prospective flight school applicants are tested for the same set of skills. The aim of our study was to assess these core competences in advanced undergraduate medical students from different medical schools. METHODS: In 2017, 67 medical students (year 5 and 6) from the universities of Hamburg, Oldenburg, and TU Munich, Germany, participated in the verified Group Assessment Performance (GAP)-Test at the German Aerospace Center (DLR) in Hamburg. All participants were rated by DLR assessment observers with a set of empirically derived behavioural checklists. This lists consisted of 6-point rating scales (1: very low occurrence to 6: very high occurrence) and included the competences leadership, teamwork, stress resistance, communication, awareness, and adherence to procedures. Medical students' scores were compared with the results of 117 admitted flight school applicants. RESULTS: Medical students showed significantly higher scores than admitted flight school applicants for adherence to procedures (p < .001, d = .63) and communication (p < .01, d = .62). They reached significantly lower ratings for teamwork (p < .001, d = .77), stress resistance (p < 0.001, d = .70), and awareness (p < .001, d = 1.31). Students in semester 10 showed significantly (p < .02, d = .58) higher scores in domain awareness compared to the final year students. On average, flight school entrance level was not reached by either group for this domain. CONCLUSIONS: Advanced medical students' low results for awareness are alarming as awareness is essential and integrative for clinical reasoning and patient safety. Further studies should elucidate and discuss whether awareness needs to be included in medical student selection or integrated into the curriculum in training units.