When to trust our learners? Clinical teachers' perceptions of decision variables in the entrustment process.

INTRODUCTION: Clinical training programs increasingly use entrustable professional activities (EPAs) as focus of assessment. However, questions remain about which information should ground decisions to trust learners. This qualitative study aimed to identify decision variables in the workplace that clinical teachers find relevant in the elaboration of the entrustment decision processes. The findings can substantiate entrustment decision-making in the clinical workplace. METHODS: Focus groups were conducted with medical and veterinary clinical teachers, using the structured consensus method of the Nominal Group Technique to generate decision variables. A ranking was made based on a relevance score assigned by the clinical teachers to the different decision variables. Field notes, audio recordings and flip chart lists were analyzed and subsequently translated and, as a form of axial coding, merged into one list, combining the decision variables that were similar in their meaning. RESULTS: A list of 11 and 17 decision variables were acknowledged as relevant by the medical and veterinary teacher groups, respectively. The focus groups yielded 21 unique decision variables that were considered relevant to inform readiness to perform a clinical task on a designated level of supervision. The decision variables consisted of skills, generic qualities, characteristics, previous performance or other information. We were able to group the decision variables into five categories: ability, humility, integrity, reliability and adequate exposure. DISCUSSION: To entrust a learner to perform a task at a specific level of supervision, a supervisor needs information to support such a judgement. This trust cannot be credited on a single case at a single moment of assessment, but requires different variables and multiple sources of information. This study provides an overview of decision variables giving evidence to justify the multifactorial process of making an entrustment decision.

Am I ready for it? Students' perceptions of meaningful feedback on entrustable professional activities.

BACKGROUND: Receiving feedback while in the clinical workplace is probably the most frequently voiced desire of students. In clinical learning environments, providing and seeking performance-relevant information is often difficult for both supervisors and students. The use of entrustable professional activities (EPAs) can help to improve student assessment within competency-based education. This study aimed to illustrate what students' perceptions are of meaningful feedback viewed as conducive in preparing for performing EPA unsupervised. METHODS: In a qualitative multicentre study we explored students' perceptions on meaningful feedback related to EPAs in the clinical workplace. Focus groups were conducted in three different healthcare institutes. Based on concepts from the literature, the transcripts were coded, iteratively reduced and displayed. RESULTS: Participants' preferences regarding meaningful feedback on EPAs were quite similar, irrespective of their institution or type of clerkship. Participants explicitly mentioned that feedback on EPAs could come from a variety of sources. Feedback must come from a credible, trustworthy supervisor who knows the student well, be delivered in a safe environment and stress both strengths and points for improvement. The feedback should be provided immediately after the observed activity and include instructions for follow-up. Students would appreciate feedback that refers to their ability to act unsupervised. CONCLUSION: There is abundant literature on how feedback should be provided, and what factors influence how feedback is sought by students. This study showed that students who are training to perform an EPA unsupervised have clear ideas about how, when and from whom feedback should be delivered.

Comparative effectiveness of a serious game and an e-module to support patient safety knowledge and awareness.

BACKGROUND: Serious games have the potential to teach complex cognitive skills in an engaging way, at relatively low costs. Their flexibility in use and scalability makes them an attractive learning tool, but more research is needed on the effectiveness of serious games compared to more traditional formats such e-modules. We investigated whether undergraduate medical students developed better knowledge and awareness and were more motivated after learning about patient-safety through a serious game than peers who studied the same topics using an e-module. METHODS: Fourth-year medical students were randomly assigned to either a serious game that included video-lectures, biofeedback exercises and patient missions (n = 32) or an e-module, that included text-based lectures on the same topics (n = 34). A third group acted as a historical control-group without extra education (n = 37). After the intervention, which took place during the clinical introduction course, before the start of the first rotation, all students completed a knowledge test, a self-efficacy test and a motivation questionnaire. During the following 10-week clinical rotation they filled out weekly questionnaires on patient-safety awareness and stress. RESULTS: The results showed patient safety knowledge had equally improved in the game group and e-module group compared to controls, who received no extra education. Average learning-time was 3 h for the game and 1 h for the e-module-group. The serious game was evaluated as more engaging; the e-module as more easy to use. During rotations, students in the three groups reported low and similar levels of patient-safety awareness and stress. Students who had treated patients successfully during game missions experienced higher self-efficacy and less stress during their rotation than students who treated patients unsuccessfully. CONCLUSIONS: Video-lectures (in a game) and text-based lectures (in an e-module) can be equally effective in developing knowledge on specific topics. Although serious games are strongly engaging for students and stimulate them to study longer, they do not necessarily result in better performance in patient safety issues.

How a teaching rotation in medical school affects graduates' subsequent careers.

INTRODUCTION: Teaching opportunities and teacher courses for medical students are increasingly offered by medical schools but little has been investigated about their long-term effect. The aim of our study was to investigate the long-term career effect of an intensive elective teaching experience for final year medical students. METHODS: We approached UMC Utrecht medical graduates who had taken a final year, 6­week full time student teaching rotation (STR) elective, 6 to 9 years after graduation, with an online survey to ask about their educational activities and obtained teaching certificates, their current roles related to education, and their appreciation of the rotation, even if this was a long time ago. In addition, we surveyed control groups of students who had not taken the STR, divided into those who had expressed interest in the STR but had not been placed and those who had not expressed such interest. RESULTS: We received responses from 50 STR graduates and 88 non-STR graduates (11 with interest and 77 without interest in the STR). STR graduates were more educationally active, had obtained more university teaching certificates and were more enthusiastic teachers. However, we could not exclude confounding, caused by a general interest in education even before the STR. CONCLUSIONS: Our findings indicate a high appreciation of the student teaching rotation and a likely but not proven long-term association between STR participation and building an educational career.

Dynamics of career choice among students in undergraduate medical courses. A BEME systematic review: BEME Guide No. 33.

INTRODUCTION: Due to the lack of a theoretically embedded overview of the recent literature on medical career decision-making, this study provides an outline of these dynamics. Since differences in educational routes to the medical degree likely affect career choice dynamics, this study focuses on medical career decision-making in educational systems with a Western European curriculum structure. METHODS: A systematic search of electronic databases (Medline, Embase) was conducted from January 2008 to November 2014. A panel of seven independent reviewers performed the data extraction, quality assessment and data synthesis using the Bland-Meurer model of medical specialty choice as a reference. RESULTS: Fifty-seven studies met the inclusion criteria for the review. Factors associated with specialty preference or career choice can be classified in five main categories: (1) medical school characteristics (e.g., curriculum structure), (2) student characteristics (e.g., age, personality), (3) student values (e.g., personal preference), (4) career needs to be satisfied (e.g., expected income, status, and work-life balance), and (5) perception of specialty characteristics (e.g., extracurricular or curricular experiences). Especially career needs and perceptions of specialty characteristics are often associated with medical career decision-making. CONCLUSION: Our results support that medical career decisions are formed by a matching of perceptions of specialty characteristics with personal needs. However, the process of medical career decision-making is not yet fully understood. Besides identifying possible predictors, future research should focus on detecting interrelations between hypothesized predictors and identify the determinants and interrelations at the various stages of the medical career decision-making process.

Variables that affect the process and outcome of feedback, relevant for medical training: a meta-review.

CONTEXT: Feedback is considered important in medical education. The literature is not clear about the mechanisms that contribute to its effects, which are often small to moderate and at times contradictory. A variety of variables seem to influence the impact of feedback on learning. The aim of this study was to determine which variables influence the process and outcomes of feedback in settings relevant to medical education. METHODS: A myriad of studies on feedback have been conducted. To determine the most researched variables, we limited our review to meta-analyses and literature reviews published in the period from January 1986 to February 2012. According to our protocol, we first identified features of the feedback process that influence its effects and subsequently variables that influence these features. We used a chronological model of the feedback process to categorise all variables found. RESULTS: A systematic search of ERIC, PsycINFO and MEDLINE yielded 1101 publications, which we reduced to 203, rejecting papers on six exclusion criteria. Of these, 46 met the inclusion criteria. In our four-phase model, we identified 33 variables linked to task performance (e.g. task complexity, task nature) and feedback reception (e.g. self-esteem, goal-setting behaviour) by trainees, and to observation (e.g. focus, intensity) and feedback provision (e.g. form, content) by supervisors that influence the subsequent effects of the feedback process. Variables from all phases influence the feedback process and effects, but variables that influence the quality of the observation and rating of the performance dominate the literature. There is a paucity of studies addressing other, seemingly relevant variables. CONCLUSIONS: The larger picture of variables that influence the process and outcome of feedback, relevant for medical education, shows many open spaces. We suggest that targeted studies be carried out to expand our knowledge of these important aspects of feedback in medical education.

Support for external validity of radiological anatomy tests using volumetric images.

RATIONALE AND OBJECTIVES: Radiology practice has become increasingly based on volumetric images (VIs), but tests in medical education still mainly involve two-dimensional (2D) images. We created a novel, digital, VI test and hypothesized that scores on this test would better reflect radiological anatomy skills than scores on a traditional 2D image test. To evaluate external validity we correlated VI and 2D image test scores with anatomy cadaver-based test scores. MATERIALS AND METHODS: In 2012, 246 medical students completed one of two comparable versions (A and B) of a digital radiology test, each containing 20 2D image and 20 VI questions. Thirty-three of these participants also took a human cadaver anatomy test. Mean scores and reliabilities of the 2D image and VI subtests were compared and correlated with human cadaver anatomy test scores. Participants received a questionnaire about perceived representativeness and difficulty of the radiology test. RESULTS: Human cadaver test scores were not correlated with 2D image scores, but significantly correlated with VI scores (r = 0.44, P < .05). Cronbach's α reliability was 0.49 (A) and 0.65 (B) for the 2D image subtests and 0.65 (A) and 0.71 (B) for VI subtests. Mean VI scores (74.4%, standard deviation 2.9) were significantly lower than 2D image scores (83.8%, standard deviation 2.4) in version A (P < .001). VI questions were considered more representative of clinical practice and education than 2D image questions and less difficult (both P < .001). CONCLUSIONS: VI tests show higher reliability, a significant correlation with human cadaver test scores, and are considered more representative for clinical practice than tests with 2D images.

Very long-term retention of basic science knowledge in doctors after graduation.

CONTEXT: Despite frequent complaints that biomedical knowledge is quickly forgotten after it has been learned, few investigations of actual long-term retention of basic science knowledge have been conducted in the medical domain. OBJECTIVES: Our aim was to illuminate the long-term retention of basic science knowledge, particularly of unrehearsed knowledge. METHODS: Using a cross-sectional study design, medical students and doctors in the Netherlands were tested for retention of basic science knowledge. Relationships between retention interval and proportion of correct answers on a knowledge test were investigated. RESULTS: The popular notion that most of basic science knowledge is forgotten shortly after graduation is not supported by our findings. With respect to the full test scores, which reflect a composite of unrehearsed and rehearsed knowledge, performance decreased from approximately 40% correct answers for students still in medical school, to 25-30% correct answers for doctors after many years of practice. When rehearsal during the retention interval is controlled for, it appears that little knowledge is lost for 1.5-2 years after it was last used; from then on, retention is best described by a negatively accelerated (logarithmic) forgetting curve. After ≥ 25 years, retention levels were in the range of 15-20%. CONCLUSIONS: Conclusions about the forgetting of unrehearsed knowledge in this study are in line with findings reported in other domains: it proceeds in accordance with the Ebbinghaus curve for meaningful material, except that in our findings the 'downward' part appears to start later than in most other studies. The limitations of the study are discussed and possible ramifications for medical education are proposed.

Assessment of competence and progressive independence in postgraduate clinical training.

CONTEXT: At present, competency-based, outcome-focused training is gradually replacing more traditional master-apprentice teaching in postgraduate training. This change requires a different approach to the assessment of clinical competence, especially given the decisions that must be made about the level of independence allowed to trainees. METHODS: This study was set within postgraduate obstetrics and gynaecology training in the Netherlands. We carried out seven focus group discussions, four with postgraduate trainees from four training programmes and three with supervisors from three training programmes. During these discussions, we explored current opinions of supervisors and trainees about how to determine when a trainee is competent to perform a clinical procedure and the role of formal assessment in this process. RESULTS: When the focus group recordings were transcribed, coded and discussed, two higher-order themes emerged: factors that determine the level of competence of a trainee in a clinical procedure, and factors that determine the level of independence granted to a trainee or acceptable to a trainee. CONCLUSIONS: From our study, it is evident that both determining the level of competence of a trainee for a certain professional activity and making decisions about the degree of independence entrusted to a trainee are complex, multi-factorial processes, which are not always transparent. Furthermore, competence achieved in a certain clinical procedure does not automatically translate into more independent practice. We discuss the implications of our findings for the assessment of clinical competence and provide suggestions for a transparent assessment structure with explicit attention to progressive independence.

Correlations of knowledge and preference of medical students for a specialty career: a case-study of youth health care.

BACKGROUND: Medical students develop interest in a specialty career during medical school based on knowledge and clinical experience of different specialties. How valid this knowledge is and how this knowledge relates to the development of preference for a specialty is not known. We studied their "subjective" knowledge of a specialty (students' reported knowledge) with "objective" knowledge of it (students actual knowledge as compared to reports of specialists) and their preference for this specialty at different stages of education, and used youth health care as a case study. METHODS: Students from all years in two medical schools (N = 2928) were asked to complete a written questionnaire including (a) a statement of their knowledge of youth health care (YHC) ("subjective knowledge"), (b) their preference for a YHC career and (c) a list of 47 characteristics of medical practice with the request to rate their applicability to YHC. A second questionnaire containing the same 47 characteristics were presented to 20 practicing youth health physicians with the request to rate the applicability to their own profession. This profile was compared to the profiles generated by individual student's answers, resulting in what we called "objective knowledge." RESULTS: Correlation studies showed that "subjective knowledge" was not related to "objective knowledge" of the YHC profession (r = 0.05), but significantly to career preference for this field (r = 0.29, P < 0.01). Preference for a YHC career hardly correlated with objective knowledge about this profession (r = 0.11, P < 0.05). Students with YHC clerkships showed no better "objective knowledge" about the profession than students without such experience. CONCLUSION: Career preference aren't always related to prior experiences, or to actual knowledge of the area. This study shows how careful we should be to trust students' opinions and preferences about specialties; they probably need much guidance in career choice.

Clinical and basic science teachers' opinions about the required depth of biomedical knowledge for medical students.

The aim of the present study was to investigate whether basic scientists and physicians agree on the required depth of biomedical knowledge of medical students at graduation. A selection of basic science and clinical teachers rated the relevance of biomedical topics for students at graduation, illustrated by 80 example items. The items were derived from ten organ systems and designed at four levels: clinical, organ, cellular and molecular. Respondents were asked to identify for each item to what extent recently graduated medical students should have knowledge about it. In addition, they were asked to indicate whether the content of the item should be included in the medical curriculum. Analysis showed that basic scientists and physicians do not diverge at the clinical level. At the organ, cellular and molecular levels however, basic scientists judge that medical students should have more active knowledge. As expected, basic scientists also indicate that more deep level content should be included. Explanations for this phenomenon will be discussed.

Analysing the concept of context in medical education.

BACKGROUND: There is increasing interest in the role of context in medical education, with the conjecture that learning in a clinical context may be helpful for later recall of knowledge. Although this may be true in a general sense, at a closer look it appears that the notion of context is not well substantiated in the medical education literature and that the concept is not clearly defined. Effects of context on learning appear to depend on type of learning task, the relationship or interaction between the context and the learning material, and motivational features of the context. Context is often implicitly regarded as a uniform concept but conceptual analysis shows that a distinction can be made in several dimensions. RESULTS: In this paper, we identify 3 different dimensions of context: a physical dimension, representing the environmental characteristics; a semantic dimension, reflecting how well the context contributes to the learning task, and a commitment dimension, representing the amount of commitment (in terms of motivation and responsibility) that is generated by the context. On these dimensions, context can be ordered from reduced (providing few cues, little meaning, little commitment) to enriched (many cues, much meaning, high commitment). CONCLUSION: This model can serve a dual purpose: first, to disentangle several aspects of educational contexts (e.g. as high in meaning but low in commitment), and second, to provide a theoretical framework to generate research on the influence of different contexts in education on students' learning.