Entrustable professional activities (EPAs) for undergraduate medical education - development and exploration of social validity.

BACKGROUND: The development of entrustable professional activities (EPAs) as a framework for work-based training and assessment in undergraduate medical education has become popular. EPAs are defined as units of a professional activity requiring adequate knowledge, skills, and attitudes, with a recognized output of professional labor, independently executable within a time frame, observable and measurable in its process and outcome, and reflecting one or more competencies. Before a new framework is implemented in a specific context, it is valuable to explore social validity, that is, the acceptability by relevant stakeholders. AIM: The aim of our work was to define Core EPAs for undergraduate medical education and further explore the social validity of the constructs. METHOD AND MATERIAL: In a nationwide collaboration, EPAs were developed using a modified Delphi procedure and validated according to EQual by a group consisting of teachers nominated from each of the seven Swedish medical schools, two student representatives, and an educational developer (n = 16). In the next step, social validity was explored in a nationwide survey. The survey introduced the suggested EPAs. For each EPA, the importance of the EPA was rated, as was the rater's perception of the present graduates' required level of supervision when performing the activity. Free-text comments were also included and analyzed. RESULTS: Ten Core EPAs were defined and validated. The validation scores for EQual ranged from 4.1 to 4.9. The nationwide survey had 473 responders. All activities were rated as "important" by most responders, ranging from 54 to 96%. When asked how independent current graduates were in performing the ten activities, 6 to 35% reported "independent". The three themes of the free text comments were: 'relevant target areas and content'; 'definition of the activities'; and 'clinical practice and learning'. CONCLUSION: Ten Core EPAs were defined and assessed as relevant for Swedish undergraduate medical education. There was a consistent gap between the perceived importance and the certainty that the students could perform these professional activities independently at the time of graduation. These results indicate that the ten EPAs may have a role in undergraduate education by creating clarity for all stakeholders.

Synthesis and perspectives from the Ottawa 2022 conference on the assessment of competence.

INTRODUCTION: The Ottawa Conference on the Assessment of Competence in Medicine and the Healthcare Professions was first convened in 1985 in Ottawa. Since then, what has become known as the Ottawa conference has been held in various locations around the world every 2 years. It has become an important conference for the community of assessment - including researchers, educators, administrators and leaders - to share contemporary knowledge and develop international standards for assessment in medical and health professions education. METHODS: The Ottawa 2022 conference was held in Lyon, France, in conjunction with the AMEE 2022 conference. A diverse group of international assessment experts were invited to present a symposium at the AMEE conference to summarise key concepts from the Ottawa conference. This paper was developed from that symposium. RESULTS AND DISCUSSION: This paper summarises key themes and issues that emerged from the Ottawa 2022 conference. It highlights the importance of the consensus statements and discusses challenges for assessment such as issues of equity, diversity, and inclusion, shifts in emphasis to systems of assessment, implications of 'big data' and analytics, and challenges to ensure published research and practice are based on contemporary theories and concepts.

Balancing acting and adapting: a qualitative study of medical students' experiences of early clinical placement.

BACKGROUND: Clinical learning experience is an important part of medical education. In the clinical learning environment, students are exposed to various aspects of medical care and may train their skills under supervision. Supervision, in which students' learning needs and the outcomes of placements are met, is essential. The aim of this study was to explore medical students' experiences of the early stages of clinical training. METHODS: In 2021, 18 individual semi-structured interviews were conducted with medical students after their first clinical placements in semester 5. The interviews were transcribed verbatim and analyzed using qualitative content analysis according to Graneim and Lundman. RESULTS: The findings resulted in an overall theme: balancing acting and adapting. Three categories described that the clinical learning environment was a big leap from campus, that personal relationships influenced learning, and that the organization of clinical placements was suboptimal. The students were encouraged to push themselves forward to practice clinical skills. This, however, did not suit all the students; the cautious ones risked becoming passive spectators. The intended learning outcomes were not frequently used; rather, the supervisors asked the students what they had learned, or the students focused on what seemed to be important on the ward. The students tried to adapt to their supervisors' working situation and not to be a burden to them. CONCLUSIONS: Our findings show that the transition from learning on campus was sometimes abrupt, as the students had to switch to a more active learning role. Ad hoc solutions in supervision occurred, which contributed to the experience that educational responsibilities were downgraded and the opportunities for clinical training varied. Rather than trying to change the circumstances, the students opted to adapt to the busy clinical learning environment.

Faculty perceptions of factors that indicate successful educational outcomes of medical students' research projects: a focus group study.

BACKGROUND: A growing number of medical schools have individual scholarly projects as a component of their curricula. The fact that all students, and not only those with research interests, have to carry out a project puts high demands on the projects and their supervision. Evidence is lacking for how to produce scholarly projects with satisfactory outcomes. This study aimed to explore the observations of faculty teachers regarding factors that predict the educational outcomes of medical students' scholarly projects. METHODS: Two focus group interviews were held with seven of the 16 faculty coordinators who were external reviewers of students' research projects. The audio-recorded interview transcripts were analyzed using qualitative content analysis. We employed a constant comparative approach to create categories firmly grounded in the participants' experiences. A successful project was defined as coordinators' perception that the stated learning outcomes were achieved, in terms of students' ability to demonstrate a scientific attitude. RESULTS: Five categories emerged from the data: Supervision, Project setup, Student characteristics, Curriculum structure, and Institutional guidance. The supervisors' experience and availability to students were mentioned as key factors for successful outcomes. Further, a clear aim and adaptation to the time frame were stated to be project-related factors that were also supervisors' responsibilities. Important student-related factors were skills related to scientific writing, taking ownership of and managing the projects, and making use of feedback. Finally, the course requirements, support, and control accomplished by faculty coordinators played important roles. CONCLUSIONS: Contributing factors to achievement of the learning outcomes were supervisors' commitment and experience, and the projects being suitable for the time frame and having a clearly stated research question. Furthermore, the students' prowess at scientific writing, adequate handling of feedback, and ability to assume ownership of the project contributed to the final outcome, as did adherence to curricular instructions.

[Portfolio - a tool for making learning and competence development visible]. / Portföljen synliggör lärandet och kompetensutvecklingen.

Portfolio used in education can be defined as a collection of documentation of performed learning activities, feedback, and progress. Currently, the documentation is electronic, hence the term e-portfolio is used. The portfolio must have a clear purpose and be aligned with the learning outcomes of the program. To be successfully implemented a portfolio must be an integral part of the education with defined tasks for both the students and the teachers. Students and teacher support in how to use the portfolio is essential especially in the beginning of the program. Learning analytics enables teachers to identify and develop support for students at risk of not achieving the outcomes.

-A cross-sectional study of clinical learning environments across four undergraduate programs using the undergraduate clinical education environment measure.

BACKGROUND: The clinical learning environment (CLE) influences students' achievement of learning outcomes and the development of their professional behaviors. However, CLEs are not always optimal for learning because of clinical productivity expectations and a lack of support from supervisors. The purpose of this study was to describe and compare students' perceptions of their CLEs across four undergraduate programs. METHODS: This study is cross-sectional. In total, 735 students who were registered in the medical, nursing, physiotherapy, and speech-language pathology (SLP) programs were invited to participate. Data were collected using an online survey, which included demographics and the Undergraduate Clinical Education Environment Measure (UCEEM). The UCEEM consists of 26 items congregated into two overarching dimensions-experiential learning and social participation-with four subscales: opportunities to learn in and through work and quality of supervision, preparedness for student entry, workplace interaction patterns and student inclusion, and equal treatment. RESULTS: In total 280 students (median age 28; range: 20-52; 72% females) returned the questionnaire. The mean total UCEEM score was 98.3 (SD 18.4; range: 91-130), with physiotherapy students giving the highest scores and medical students the lowest. The mean scores for the dimensions experiential learning and social participation for all the students were 62.8 (SD 13.6; range 59-85) and 35.5 (SD 6.2; range 13-45), respectively. Medical students rated the lowest for all subscales. The items receiving the highest ratings concerned equal treatment, whereas those receiving the lowest ratings concerned supervisors' familiarity with the learning objectives. There were few statistically significant differences between the semesters within each program. CONCLUSIONS: The students generally hold positive perceptions toward their CLEs. However, the students from the medical and nursing programs rated their learning environment lower than did the students from the physiotherapy and SLP programs. Importantly, in several aspects, the medical students provided significantly lower ratings for their CLE compared with the students from the other programs. The medical students' low ratings for their supervisors' familiarity with the learning objectives underscore the need to ensure that the prerequisites for optimal supervision are met.

A prospective study for evaluation of structural and clinical validity of the Eating Assessment Tool.

BACKGROUND: The Eating Assessment Tool is a self-rating questionnaire developed to assess symptom severity and treatment efficacy in swallowing disorders. The aim of this study was to investigate the structural validity of the Eating Assessment Tool and whether individual item scores differed between dysphagia secondary to neurological and structural/esophageal disorders, respectively. METHODS: This is a prospective cross-sectional questionnaire study. In total, 200 community-dwelling adults with suspected dysphagia referred for fiberoptic endoscopic examination of swallowing at Karolinska University Hospital, Stockholm, Sweden, completed the S-EAT-10. Patients' medical charts were reviewed in order to establish the type of dysphagia. Principal axis factoring was conducted to examine structural validity, and Mann-Whitney U tests were used to study differences in the S-EAT-10 score patterns between different types of dysphagia. RESULTS: One single factor explained 54% of the total variance in EAT-10 item scores. All ten items loaded substantially or strongly on this factor, supporting the single-factor solution (Cronbach's alpha = 0.90). Structural/esophageal dysphagia was associated with higher scores on six items and with a higher total EAT-10 score. CONCLUSIONS: The EAT-10 yields a unidimensional index of symptom severity in patients with dysphagia. Individual item scores reflect typical symptoms in neurogenic and structural/esophageal dysphagia, supporting its clinical relevance.

[It's time to define the Swedish Doctor? Lessons learnt from the evaluation of the Finnish undergraduate medical education]. / Dags att definiera vad en svensk doktor ska kunna - Lärdomar av utvärderingen av den finska läkarutbildningen.

The proposal for a new Swedish 6-year undergraduate medical education entails that the Master of Science in Medicine provides the basis for the licence to practice. During 2017-2018, the Finnish National Education Evaluation Center (FINEEC) conducted a nationwide evaluation of their 6-year medical education. Experiences from their evaluation may be useful in the development of the new Swedish undergraduate education. The outcome important to consider in Sweden is defining common core competencies for all newly graduated doctors, a framework for "Swedish Doctor". A curriculum mapping tool to ensure constructive alignment and an organization to oversee how the curriculum is implemented is essential. Further, the universities have to ensure that all students get training in essential clinical skills and that those skills are assessed. Development of the national collaboration similar to that of Finland is encouraged.

[EPA (Entrustable professional activities) - an international approach to define key tasks that a doctor should be able to perform]. / EPA ­ en modell för att träna och bedöma dagligt läkarjobb.

Entrustable Professional Activities (EPAs) define observable key tasks that a doctor should be able to do independently after having completed a certain level of education. Progression is facilitated by frequent assessments of the defined activities, with increased degree of independence. The observations should be supported by constructive feed-forward. EPA is currently used in several countries both in undergraduate and post graduate education. In the context of pending changes in Swedish legislation regarding undergraduate education qualifications ordinance and medical licensing, a Swedish national work group has begun to define Swedish EPA for the future undergraduate education.

Stress among medical students during clinical courses: a longitudinal study using contextual activity sampling system.

OBJECTIVES: To investigate medical students' experiences of stress and other emotions related to their professional roles, as defined by the CanMEDS framework, by using the Contextual Activity Sampling System (CASS). METHODS: Ninety-eight medical students agreed to participate of whom 74 completed this longitudinal cohort study. Data was collected between 6th and 8th term via CASS methodology: A questionnaire was e-mailed to the participants every 3rd week(21questionnaires/measurements) during clinical rotations and scientific project work term. Emotions were measured by a 7-point Likert scale (e.g., maximum stress = 7). Answers were registered through mobile technology. We used a linear mixed-model regression approach to study the association between stress over time in relation to socio-demographic and learning activities related to CanMEDS roles. RESULTS: Participants completed 1390 questionnaires. Mean stress level over all time points was 3.6. Stress was reported as highest during the scientific project term. Learning activities related to 'Communicator,' 'Collaborator,' 'Scholar,' 'Manager' and 'Professional' were associated with increased stress, e.g. 'Scholar' increased stress with 0.5 points (t(1339)=3.91, p<0.001). A reduced level of stress was associated with 'Health Advocate' of 0.39 points (t(1338)=-2.15, p=0.03). No association between perceived stress and demographic factors, such as gender or age was found. CONCLUSIONS: An association between different learning activities related to CanMEDS Roles and feelings of stress were noted. The CASS methodology was found to be useful when observing learning experiences and might support educational development by identifying course activities linked to stress.

Does reality meet expectations? An analysis of medical students' expectations and perceived learning during mandatory research projects.

BACKGROUND: Although much has been written about structure and outcomes of medical students' curricular research projects, less attention has been paid to the expectations on such projects. In order to foster students' scientific understanding and improve the quality of mandatory research projects, we compared students' pre-course expectations with their post-course insights regarding learning and transferable skills. METHODS: A prospective cross-sectional questionnaire study. All students registered on a mandatory 20-week research project course in 2011-2013 were e-mailed questionnaires in the beginning and after the course asking them to rate statements on expectations and perceived learning on a 5-point Likert scale. Of 652 students, 358 (mean age 26 years; range 21-49; 63% females) returned both questionnaires, corresponding to a response rate of 55%. RESULTS: The ratings for expectations as well as perceived learning were highest for learning to search and critically appraise literature. The greatest pre- and post-course differences were indicated for participation in scientific discussions and oral communication. Surprisingly, both pre- and post-course ratings were low for research ethics. The highest post-course ratings regarding skills for future working life were given to items pertaining to understanding the scientific basis of medicine, ability to follow the development of knowledge and to critically integrate knowledge. Female students had higher expectations than male students. Those with a previous university degree had lower ratings of expectations and perceived learning. Students with basic science projects reported higher expectations and higher learning compared to students with other projects. Previous research experience had no significant influence on expectations nor learning. The correlations between post-course ratings of learning and skills showed that problem-solving ability had a relatively high correlation with all skills. CONCLUSIONS: Students had high expectations and perceived the course improved crucial practical skills. However, expectations were not quite met regarding aspects of scientific communication, and hypothesis formulation, likely because these require more extensive practice and feedback. Students should be actively involved in ethical discussions and oral communication should be trained repeatedly as it is an important task of doctors to communicate scientific information to patients and non-experts.

[Examination of future colleagues: We need an assessment culture]. / Att examinera blivande kollegor: Vi behöver en bedömningskultur.

How do we know that the newly graduated student have the competencies required to work as a doctor? Assessments are an integral part of a medical program and are also required to be specified in the syllabi. High-quality assessments are in line with both learning outcomes and content of the courses. Complex learning outcomes require different assessment methods, multiple examiners, and psychometric analysis of the results. Further, the assessment criteria have to be known in advance, and feedback to students should be mandatory. It has to be remembered that the most important tool is the assessor himself, rather than the tools used in the assessment. Teachers need education in assessment. A coherent assessment system helps to create blueprint templates and make it possible to assess progression during the education. The link between examinations and patient safety is critical. We will not get a good education and safer care for our patients without creating an assessment culture.

Medical students' perceptions of their learning environment during a mandatory research project.

OBJECTIVES: To explore medical students´ perceptions of their learning environment during a mandatory 20-week scientific research project. METHODS: This cross-sectional study was conducted between 2011 and 2013. A total of 651 medical students were asked to fill in the Clinical Learning Environment, Supervision, and Nurse Teacher (CLES+T) questionnaire, and 439 (mean age 26 years, range 21-40, 60% females) returned the questionnaire, which corresponds to a response rate of 67%. The Mann-Whitney U test or the Kruskal-Wallis test were used to compare the research environments. RESULTS: The item My workplace can be regarded as a good learning environment correlated strongly with the item There were sufficient meaningful learning situations (r= 0.71, p<0.001). Overall satisfaction with supervision correlated strongly with the items interaction (r=0.78, p < 0.001), feedback (r=0.76, p<0.001), and a sense of trust (r=0.71, p < 0.001).  Supervisors´ failures to bridge the gap between theory and practice or to explain intended learning outcomes were important negative factors.  Students with basic science or epidemiological projects rated their learning environments higher than did students with clinical projects (χ2(3, N=437)=20.29, p<0.001). CONCLUSIONS: A good research environment for medical students comprises multiple meaningful learning activities, individual supervision with continuous feedback, and a trustful atmosphere including interactions with the whole staff.  Students should be advised that clinical projects might require a higher degree of student independence than basic science projects, which are usually performed in research groups where members work in close collaboration.

Medical students' research productivity and career preferences; a 2-year prospective follow-up study.

BACKGROUND: Linking undergraduate medical education to scientific research is necessary for the quality of future health care, and students´ individual research projects are one way to do so. Assessment of the impact of such projects is of interest for both educational and research-oriented segments of medical schools. Here, we examined the scholarly products and medical students' career preferences 2 years after a mandatory research project course. METHODS: A prospective cross-sectional questionnaire study. All 581 students registered on a 20-week research project course between September 2010 through September 2012 were e-mailed a questionnaire 2 years after completing the course. RESULTS: In total, 392 students (mean age 27 years; 60% females) responded (67% response rate). 59 students (15%) were co-authors on a scientific paper published in an international journal, 6 students had published in a national journal, and 57 students had co-authored a paper submitted for publication. Totally, 122 scientific papers had been submitted. Moreover, 67 (17%) students had given 107 oral or poster presentations nationally or internationally during the follow-up. Career-wise, 36 students (9%) had been registered as PhD students and an additional 127 students (34%) were planning to register. Those who did not plan doctoral studies were significantly older (p = 0.013) than those who did. However, 35% reported that they would in the coming 5 years prefer to work as clinicians only, and this group was significantly younger than those who envisaged participation in research. There were no significant gender differences. CONCLUSIONS: Approximately a third of the students had authored papers and/or public presentations, and a similar fraction had career plans involving a PhD degree. The results indicate that the project course had a positive impact on continued supervisor-student collaboration on a professional level, but also that strategies to encourage young doctors to perform clinical research may be needed.

Medical students' experiences of their own professional development during three clinical terms: a prospective follow-up study.

BACKGROUND: A modern competency-based medical education is well implemented globally, but less is known about how the included learning activities contribute to medical students' professional development. The aim of this study was to explore Swedish medical students' perceptions of the offered learning activities and their experiences of how these activities were connected to their professional development as defined by the CanMEDS framework. METHODS: A prospective mixed method questionnaire study during three terms (internal medicine, scientific project, and surgery) in which data were collected by using contextual activity sampling system, i.e., the students were sent a questionnaire via their mobile phones every third week. All 136 medical students in the 6th of 11 terms in the autumn of 2012 were invited to participate. Seventy-four students (54%) filled in all of the required questionnaires (4 per term) for inclusion, the total number of questionnaires being 1335. The questionnaires focused on the students' experiences of learning activities, especially in relation to the CanMEDS Roles, collaboration with others and emotions (positive, negative, optimal experiences, i.e., "flow") related to the studies. The quantitative data was analysed statistically and, for the open-ended questions, manifest inductive content analysis was used. RESULTS: Three of the CanMEDs Roles, Medical Expert, Scholar, and Communicator, were most frequently reported while the four others, e.g., the role Health Advocate, were less common. Collaboration with students from other professions was most usual during the 8th term. Positive emotions and experience of "flow" were most often reported during clinical learning activities while the scientific project term was connected with more negative emotions. CONCLUSIONS: Our results showed that it is possible, even during clinical courses, to visualise the different areas of professional competence defined in the curriculum and connect these competences to the actual learning activities. Students halfway through their medical education considered the most important learning activities for their professional development to be connected with the Roles of Medical Expert, Scholar, and Communicator. Given that each of the CanMEDS Roles is at least moderately important during undergraduate medical education, the entire spectrum of the Roles should be emphasised and developed during the clinical years.

Long-Term Follow-Up of Patients with Spasmodic Dysphonia and Improved Voice despite Discontinuation of Treatment.

OBJECTIVE: To evaluate voice function in patients with adductor spasmodic dysphonia (AdSD) who discontinued botulinum toxin (BTX) treatment because they felt that their voice had improved sufficiently. PATIENTS AND METHODS: Twenty-eight patients quit treatment in 2004, of whom 20 fulfilled the inclusion criteria for the study, with 3 subsequently excluded because of return of symptoms, leaving 17 patients (11 males, 6 females) included in this follow-up study. A questionnaire concerning current voice function and the Voice Handicap Index were completed. Audio-perceptual voice assessments were done by 3 listeners. The inter- and intrarater reliabilities were r > 0.80. RESULTS: All patients had a subjectively good stable voice, but with differences in their audio-perceptual voice assessment scores. Based on the pre-/posttreatment auditory scores on the overall degree of AdSD, patients were divided into 2 subgroups showing more and less improvement, with 10 and 7 patients, respectively. The subgroup with more improvement had shorter duration from the onset of symptoms until the start of BTX treatment, and included 7 males compared to only 4 males in the subgroup with less improvement. CONCLUSION: It seems plausible that the symptoms of spasmodic dysphonia may decrease over time. Early intervention and male gender seem to be important factors for long-term reduction of the voice symptoms of AdSD.

Validation of the Swedish translation of eating assessment tool (S-EAT-10).

Conclusion The Swedish Eating Assessment Tool (S-EAT-10) is a reliable and valid self-administered tool in assessment of dysphagia in adult Swedish patients with high internal consistency, reliability, and discriminative validity. The normative data show that a score of 3 or more is abnormal. S-EAT-10 is recommended to be used in preliminary diagnostics of dysphagia. Objective To translate and adapt the EAT-10 for use in the Swedish patient population, and to present norms and measures of discriminative validity and reliability of a Swedish version of the Eating Assessment Tool-10 (S-EAT-10). Methods Prospective consecutive clinical study. In total, 134 community-dwelling adult respondents/controls without dysphagia completed the S-EAT-10, as did 119 patients referred for fiberendoscopic evaluation of swallowing at Karolinska University Hospital, Stockholm, Sweden. Patient vs control status was used as the criterion for discriminative validity assessment by logistic regression analysis. Results The mean S-EAT-10 score was 0.2 (range = 0-3) for controls and 18 (range = 0-38) for patients. Based on a cut-off score of ≥ 3 which was considered to be reflective of abnormalities, sensitivity was 98.5% and specificity 94.1%. Internal consistency reliability was high (Cronbach's alpha = 0.88), as was test-re-test reliability (ICC = 0.90).

What is the reward? Medical students' learning and personal development during a research project course.

BACKGROUND: Until recently, the outcome of medical students' research projects has mainly been assessed in terms of scientific publications, whereas other results important for students' development have been less studied. The aim of this study was to investigate medical students' experiences of learning as an outcome of the research project course. METHOD: Written reflections of 50 students were analyzed by manifest inductive content analysis. RESULTS: Three categories emerged: 'thinking as a scientist', 'working as a scientist', and 'personal development'. Students became more aware about the nature of knowledge, how to generate new knowledge, and developed skills in scientific thinking and critical appraisal. Unexpectedly, effects on personal characteristics, such as self-confidence, self-discipline, independence, and time management skills were also acknowledged. CONCLUSIONS: We conclude that individual research projects enhance research-specific skills and competencies needed in evidence-based clinical work and are beneficial for personal and professional development.

[The supervisor has a crucial role in the medical student's degree projects. Experiences from seven semesters at Karolinska Institutet]. / Handledaren har avgörande roll för läkar-studentens examensarbete--KI:s erfarenheter från sju terminer.

In Sweden degree projects have a central role in evaluation of higher education, wherefore significant resources are spent on developing students' research competence. The undergraduate medical program at Karolinska Institutet introduced its degree project course in 2010. This paper gives an overview of the course and summarizes experiences from the first seven terms. In order to finalize their projects within one term, most students need substantial support. A highly structured course and frequent progress monitoring are advantageous. Other crucial factors are the quality of the supervision and students' verbal skills as well as support in scientific writing. In addition, increased awareness of the learning outcomes already at the beginning of the course may help students to achieve the expected results. Finally, students need to recognize their own responsibility for learning.