Dual purposes by design: exploring alignment between residents' and academic advisors' documents in a longitudinal program.

Longitudinal academic advising (AA) and coaching programs are increasingly implemented in competency based medical education (CBME) to help residents reflect and act on the voluminous assessment data they receive. Documents created by residents for purposes of reflection are often used for a second, summative purpose-to help competence committees make decisions-which may be problematic. Using inductive, thematic analysis we analyzed written comments generated by 21 resident-AA dyads in one large internal medicine program who met over a 2 year period to determine what residents write when asked to reflect, how this aligns with what the AAs report, and what changes occur over time (total 109 resident self-reflections and 105 AA reports). Residents commented more on their developing autonomy, progress and improvement than AAs, who commented far more on performance measures. Over time, residents' writing shifted away from intrinsic roles, patient care and improvement towards what AAs focused on, including getting EPAs (entrustable professional activities), studying and exams. For EPAs, the emphasis was on getting sufficient numbers rather than reflecting on what residents were learning. Our findings challenge the practice of dual-purposing documents, by questioning the blurring of formative and summative intent, the structure of forms and their multiple conflicting purposes, and assumptions about the advising relationship over time. Our study suggests a need to re-evaluate how reflective documents are used in CBME programs. Further research should explore whether and how documentation can best be used to support resident growth and development.

"Patients Are the People Who Teach Me the Most": Exploring the Development of Communication Skills During Internal Medicine Residency.

Background: Physician-patient communication training is a vital component of medical education, yet physicians do not always achieve the communication expertise expected of them. Despite extensive literature on the efficacy of various training interventions, little is known about how residents believe they learn to communicate. Objective: To understand residents' perspectives on the development of their communication skills. Methods: Between November 2020 and January 2021 recruitment emails were sent to all 225 internal medicine residents at the University of Toronto; one-on-one interviews were conducted with 15 residents. Participants were asked to reflect on communication skills development. Interviews were conducted and analyzed using constructivist grounded theory. Results: Participants credited the majority of their skills development to unsupervised interactions with patients, without explicit guidance from an attending physician. Attendings' contributions were primarily seen through role modeling, with little perceived learning coming from feedback on observed interactions. This was partly explained by residents' proclivity to alter their communication styles when observed, rendering feedback less relevant to their authentic practice, and by receiving generically positive feedback lacking in constructive features. Time constraints led to communication styles that prioritized efficiency at the cost of patient-centeredness. Conclusions: These findings suggest that current models of communication training and assessment may fall short due to overreliance on observation by attendings and examiners, which may fail to unearth the authentic and largely self-taught communication behaviors of residents. Further research is required to ascertain the feasibility and potential value of other forms of communication training and assessment, such as through patient feedback.

Supervising the senior medical resident: Entrusting the role, supporting the tasks.

INTRODUCTION: Postgraduate competency-based medical education has been implemented with programmatic assessment that relies on entrustment-based ratings. Yet, in less procedurally oriented specialties such as internal medicine, the relationship between entrustment and supervision remains unclear. We undertook the current study to address how internal medicine supervisors conceptualise entrusting senior medical residents while supervising them on the acute care wards. METHODS: Guided by constructivist grounded theory, we interviewed 19 physicians who regularly supervised senior internal medicine residents on inpatient wards at three Canadian universities. We developed a theoretical model through iterative cycles of data collection and analysis using a constant comparative process. RESULTS: On the internal medicine ward, the senior resident role is viewed as a fundamentally managerial and rudimentary version of the supervisor's role. Supervisors come to trust their residents in the senior role through an early 'hands-on' period of assessment followed by a gradual withdrawal of support to promote independence. When considering entrustment, supervisors focused on entrusting a particular scope of the senior resident role as opposed to entrustment of individual tasks. Irrespective of the scope of the role that was entrusted, supervisors at times stepped in and stepped back to support specific tasks. CONCLUSION: Supervisors' stepping in and stepping back to support individual tasks on the acute care ward has an inconsistent relationship to their entrustment of the resident with a particular scope of the senior resident role. In this context, entrustment-based assessment would need to capture more of the holistic perspective of the supervisor's entrustment of the senior resident role. Understanding the dance of supervision, from relatively static overall support of the resident in their role, to fluidly stepping in and out for specific patient care tasks, allows us insight into the affordances of the supervisory relationship and how it may be leveraged for assessment.

Beyond the ratings: gender effects in written comments from clinical teaching assessments.

Assessment of clinical teachers by learners is problematic. Construct-irrelevant factors influence ratings, and women teachers often receive lower ratings than men. However, most studies focus only on numeric scores. Therefore, the authors analyzed written comments on 4032 teacher assessments, representing 282 women and 448 men teachers in one Department of Medicine, to explore for gender differences. NVivo was used to search for 61 evidence- and theoretically-based terms purported to reflect teaching excellence, which were analyzed using 2 × 2 chi-squared tests. The Linguistic Index and Word Count (LIWC) was used to categorize comment data, which were analyzed using linear regressions. The only significant difference in NVivo was that men were more likely than women to have the word "available" in a comment (OR 1.4, p < .05). A subset of LIWC variables showed significant gender differences, but all effects were modest. Men teachers had more positive emotion words written about them, while negative emotion words appeared equally. Significant differences occurred more often between the men and women residents who wrote the comments, rather than those attributed to the gender of the teachers. For example, women residents used more social and gender-related words (ß 1.87, p < 0.001) and fewer words related to power or achievement (ß -3.78, p < 0.001) than men residents. Profound gender differences were not found in teacher assessment comments in this large, diverse academic department of medicine, which differs from other studies. The authors explore possible reasons including differences in departmental culture and issues related to the methods used.

The Senior Medical Resident's New Role in Assessment in Internal Medicine.

PURPOSE: With the introduction of competency-based medical education, senior residents have taken on a new, formalized role of completing assessments of their junior colleagues. However, no prior studies have explored the role of near-peer assessment within the context of entrustable professional activities (EPAs) and competency-based medical education. This study explored internal medicine residents' perceptions of near-peer feedback and assessment in the context of EPAs. METHOD: Semistructured interviews were conducted from September 2019 to March 2020 with 16 internal medicine residents (8 first-year residents and 8 second- and third-year residents) at the University of Toronto, Toronto, Ontario, Canada. Interviews were conducted and coded iteratively within a constructivist grounded theory approach until sufficiency was reached. RESULTS: Senior residents noted a tension in their dual roles of coach and assessor when completing EPAs. Senior residents managed the relationship with junior residents to not upset the learner and potentially harm the team dynamic, leading to the documentation of often inflated EPA ratings. Junior residents found senior residents to be credible providers of feedback; however, they were reticent to find senior residents credible as assessors. CONCLUSIONS: Although EPAs have formalized moments of feedback, senior residents struggled to include constructive feedback comments, all while knowing the assessment decisions may inform the overall summative decision of their peers. As a result, EPA ratings were often inflated. The utility of having senior residents serve as assessors needs to be reexamined because there is concern that this new role has taken away the benefits of having a senior resident act solely as a coach.

Resident Perceptions of Assessment and Feedback in Competency-Based Medical Education: A Focus Group Study of One Internal Medicine Residency Program.

PURPOSE: As key participants in the assessment dyad, residents must be engaged with the process. However, residents' experiences with competency-based medical education (CBME), and specifically with entrustable professional activity (EPA)-based assessments, have not been well studied. The authors explored junior residents' perceptions regarding the implementation of EPA assessment and feedback initiatives in an internal medicine program. METHOD: From May to November 2018, 5 focus groups were conducted with 28 first-year internal medicine residents from the University of Toronto, exploring their experiences with facilitators and barriers to EPA-based assessments in the first years of the CBME initiative. Residents were exposed to EPA-based feedback tools from early in residency. Themes were identified using constructivist grounded theory to develop a framework to understand the resident perception of EPA assessment and feedback initiatives. RESULTS: Residents' discussions reflected a growth mindset orientation, as they valued the idea of meaningful feedback through multiple low-stakes assessments. However, in practice, feedback seeking was onerous. While the quantity of feedback had increased, the quality had not; some residents felt it had worsened, by reducing it to a form-filling exercise. The assessments were felt to have increased daily workload with consequent disrupted workflow and to have blurred the lines between formative and summative assessment. CONCLUSIONS: Residents embraced the driving principles behind CBME, but their experience suggested that changes are needed for CBME in the study site program to meet its goals. Efforts may be needed to reconcile the tension between assessment and feedback and to effectively embed meaningful feedback into CBME learning environments.

Tensions in Assessment: The Realities of Entrustment in Internal Medicine.

PURPOSE: A key unit of assessment in competency-based medical education (CBME) is the entrustable professional activity. The variations in how entrustment is perceived and enacted across specialties are not well understood. This study aimed to develop a thorough understanding of the process, concept, and language of entrustment as it pertains to internal medicine (IM). METHOD: Attending supervisors of IM trainees on the clinical teaching unit were purposively sampled. Sixteen semistructured interviews were conducted and analyzed using constructivist grounded theory. The study was conducted at the University of Toronto from January to September 2018. RESULTS: Five major themes were elucidated. First, the concepts of entrustment, trust, and competence are not easily distinguished and sometimes conflated. Second, entrustment decisions are not made by attendings, but rather are often automatic and predetermined by program or trainee level. Third, entrustment is not a discrete, point-in-time assessment due to longitudinality of tasks and supervisor relationships with trainees. Fourth, entrustment scale language does not reflect attendings' decision making. Fifth, entrustment decisions affect the attending more than the resident. CONCLUSIONS: A tension arises between the need for a common language of CBME and the need for authentic representation of supervision within each specialty. With new assessment instruments required to operationalize the tenets of CBME, it becomes critically important to understand the nuanced and specialty-specific language of entrustment to ensure validity of assessments.

Understanding the Use and Perceived Impact of a Medical Podcast: Qualitative Study.

BACKGROUND: Although podcasts are increasingly being produced for medical education, their use and perceived impact in informal educational settings are understudied. OBJECTIVE: This study aimed to explore how and why physicians and medical learners listen to The Rounds Table (TRT), a medical podcast, as well as to determine the podcast's perceived impact on learning and practice. METHODS: Web-based podcast analytics were used to collect TRT usage statistics. A total of 17 medical TRT listeners were then identified and interviewed through purposive and convenience sampling, using a semistructured guide and a thematic analysis, until theoretical sufficiency was achieved. RESULTS: The following four themes related to podcast listenership were identified: (1) participants thought that TRT increased efficiency, allowing them to multitask, predominantly using mobile listening platforms; (2) participants listened to the podcast for both education and entertainment, or "edutainment"; (3) participants thought that the podcast helped them keep up to date with medical literature; and (4) participants considered TRT to have an indirect effect on learning and clinical practice by increasing overall knowledge. CONCLUSIONS: Our results highlight how a medical podcast, designed for continuing professional development, is often used informally to promote learning. These findings enhance our understanding of how and why listeners engage with a medical podcast, which may be used to inform the development and evaluation of other podcasts.

Unearthing Faculty and Trainee Perspectives of Feedback in Internal Medicine: the Oral Case Presentation as a Model.

BACKGROUND: The case presentation is a fundamental activity used in both patient care and trainee education, partly due to feedback from supervisor to trainee. Although feedback in medical education is well studied, prior studies have not focused on the perceptions of feedback by Internal Medicine supervisors and trainees as it relates to clinical activities like the case presentation. METHODS: Semi-structured interviews were conducted with eight Internal Medicine physicians, and 18 Internal Medicine trainees (5 medical students, 13 residents) at the University of Toronto. Purposive sampling was used. Interviews were conducted and coded iteratively within a constructivist grounded theory approach until saturation was reached. RESULTS: Supervisors and trainees recognized feedback as an important part of the case presentation that can be (1) explicit, labeled feedback or (2) implicit, unlabeled feedback. Both trainees and supervisors perceived that not enough feedback occurs, likely stemming from a hesitancy by supervisors to label implicit feedback, calling it an interruption instead. Although trainees were keenly aware of non-verbal feedback from their supervisors as implicit feedback, they often interpreted explicit constructive feedback negatively. Interestingly, the same feedback from senior residents was regarded as highly educational, as it was uncoupled from assessment. CONCLUSION: Feedback occurs more frequently in case presentations than previously described, particularly in an implicit, unlabeled format. Even though under-recognized, trainees identify and utilize implicit feedback from supervisors, and coaching from senior residents, to develop learned behaviors. This is reassuring in the age of Competency-Based Medical Education, as feedback has an essential role in workplace-based assessment and promotion.

Overshadowed by Assessment: Understanding Trainee and Supervisor Perspectives on the Oral Case Presentation in Internal Medicine Workplace-Based Assessment.

PURPOSE: The oral case presentation (OCP) is an essential part of daily clinical practice in internal medicine (IM) and a key competency in medical education. It is not known how supervisors and trainees perceive OCPs in workplace-based learning and assessment. METHOD: Using a constructivist grounded theory approach, 26 semistructured interviews were held with trainees and supervisors (18 clinical clerks and first- through third-year postgraduate trainees, and 8 supervisors) on the IM clinical teaching unit at the University of Toronto, 2015-2016. Interviews focused on how the OCP was viewed by both trainees and supervisors in clinical practice as a tool for patient care, learning, and assessment. Iterative, constant comparative techniques were used to analyze the interviews and develop a framework to understand trainee and supervisor perspectives. RESULTS: Supervisors and trainees viewed the OCP as an important part of informal trainee assessment in IM. Supervisors used OCPs to understand the patient through trainee-demonstrated skills including the use of narratives, information synthesis, and management of uncertainty. However, because of awareness of assessment, trainees sought to control the OCP, viewing it as a performance demonstrating their competence, mediated by senior residents and tailored to supervisor preferences. CONCLUSIONS: Preoccupied with assessment around OCPs, trainees often lost sight of the valuable learning taking place. Use of OCPs in assessment necessitates optimization of the educational activity for trainees. Providing explicit direction to both trainees and supervisors, defining expectations, and clarifying the assessment activity of the OCP can optimize the encounter for best educational practice.

Resident and attending perceptions of direct observation in internal medicine: a qualitative study.

OBJECTIVES: Direct observation is the foundation of assessment and learning in competency-based medical education (CBME). Despite its importance, there is significant uncertainty about how to effectively implement frequent and high-quality direct observation. This is particularly true in specialties where observation of non-procedural skills is highly valued and presents unique challenges. It is therefore important to understand perceptions of direct observation to ensure successful acceptance and implementation. In this study, we explored perceptions of direct observation in internal medicine. METHODS: We interviewed internal medicine attending physicians (n = 9) and residents (n = 8) at the University of Toronto, purposively sampled for diversity. Using a constructivist grounded theory approach, constant comparative analysis was performed to develop a framework to understand perceptions of direct observation on the clinical teaching units. RESULTS: Participants articulated a narrow perception of what constitutes direct observation, in contrast to their own descriptions of skills that were observed. This resulted in the perception that certain valuable skills that participants felt were routinely observed were nonetheless not 'directly observable', such as clinical reasoning, observed through case presentations and patient care discussions. Differentiating direct observation from informal observation led to overestimation of the time and resource requirements needed to enhance direct observation, which contributed to scepticism and lack of engagement related to CBME implementation. CONCLUSIONS: In an internal medicine training programme, perceptions of what constitutes direct observation can lead to under-recognition and hinder acceptance in workplace-based assessment and learning. Our results suggest a reframing of 'direct observation' for residents and attending physicians, by explicitly identifying desired skills in non-procedurally-based specialties. These findings may help CBME-based training programmes improve the process of direct observation, leading to enhanced assessment and learning.

Evaluating the effect of instruction and practice schedule on the acquisition of ECG interpretation skills.

INTRODUCTION: Evidence of the benefit of distributed instruction and interleaved practice comes from studies using simple materials (e. g. word pairs). Furthermore, there is currently no evidence of the combined impact of these strategies in undergraduate medical education. The present study evaluated the impact of varying both instruction and practice schedules for the acquisition of ECG interpretation skills. METHODS: We conducted a 2 × 2 factorial study with two levels of instruction (massed and distributed) and two levels of practice (interleaved and blocked). A three-module introductory course in ECG interpretation was delivered to 80 first year medical undergraduate students. Students were assigned to one of four Instruction-Practice conditions: Massed-Interleaved, Massed-Blocked, Distributed-Interleaved and Distributed-Blocked. Learning was evaluated by a multiple choice quiz at the end of each module and a final multiple choice quiz at the end of the course. RESULTS: End of module mean scores showed that distributed instruction was consistently superior to massed instruction (52% vs 42%, p < 0.01). However, there was no effect of practice and no interaction between teaching and practice methods. The delayed final test scores revealed an advantage for blocked over mixed practice (34% vs 24%, p < 0.05) and distributed over massed instruction (34% vs 24%, p < 0.05). DISCUSSION: These results suggest that these popular strategies may have varying effects with complex learning materials. Further research is required to understand how these strategies affect the learning of simple and very complex skills.

What Makes a Great Resident Teacher? A Multicenter Survey of Medical Students Attending an Internal Medicine Conference.

BACKGROUND: Residents have a critical role in the education of medical students and have a unique teaching relationship because of their close proximity in professional development and opportunities for direct supervision. Although there is emerging literature on ways to prepare residents to be effective teachers, there is a paucity of data on what medical students believe are the attributes of successful resident teachers. OBJECTIVE: We sought to define the qualities and teaching techniques that learners interested in internal medicine value in resident teachers. METHODS: We created and administered a resident-as-teacher traits survey to senior medical students from 6 medical schools attending a resident-facilitated clinical conference at McMaster University. The survey collected data on student preferences of techniques employed by resident teachers and qualities of a successful resident teacher. RESULTS: Of 90 student participants, 80 (89%) responded. Respondents found the use of clinical examples (78%, 62 of 80) and repetition of core concepts (71%, 58 of 80) highly useful. In contrast, most respondents did not perceive giving feedback to residents, or receiving feedback from residents, was useful to their learning. With respect to resident qualities, respondents felt that a strong knowledge base (80%, 64 of 80) and tailoring teaching to the learner's level (83%, 66 of 80) was highly important. In contrast, high expectations on the part of resident supervisors were not valued. CONCLUSIONS: This multicenter survey provides insight into the perceptions of medical students interested in internal medicine on the techniques and qualities that characterize successful resident teachers. The findings may be useful in the future development of resident-as-teacher curricula.