Trainee selection of tasks in postgraduate medical education: Is there a role for 'cherry-picking' to optimise learning?

PURPOSE: Learning is optimised when postgraduate trainees engage in clinical tasks in their zone of proximal development (ZPD). However, workplace learning environments impose additional non-learning goals and additional tasks that may lead to trainees engaging in tasks that do not fall within their ZPD. We do not fully understand how trainees select clinical tasks in the workplace learning environment. If we knew the goals and factors they consider when selecting a task, we could better equip trainees with strategies to select tasks that maximise learning. We explored how postgraduate trainees select clinical tasks using echocardiography interpretation as a model. METHODS: Canadian General Cardiology residents and Echocardiography fellows were invited to participate in semi-structured interviews. Aligning with a theory-informed study, two independent researchers used a deductive, directed content analysis approach to identify codes and themes. RESULTS: Eleven trainees from seven Canadian universities participated (PGY4 = 4, PGY5 = 3, PGY6 = 1 and echocardiography fellows = 3). Goals included learning content, fulfilling assessment criteria and contributing to clinical demands. Trainees switched between goals throughout the day, as it was too effortful for them to engage in tasks within their ZPD at all times. When trainees had sufficient mental effort available, they selected higher complexity tasks that could advance learning content. When available mental effort was low, trainees selected less complex tasks that fulfilled numerically based assessment goals or contributed to clinical demands. Trainees predominantly used perceived complexity of the echocardiogram as a factor to select tasks to achieve their desired goals. CONCLUSION: Postgraduate trainees select tasks within their ZPD that enable them to maximise learning when they perceive to have sufficient mental effort available and workplace affordances are adequate. These findings can inform individual and systemic strategies to maximise learning when selecting tasks.

Simulation vs workplace-based assessment in resuscitation: a cross-specialty descriptive analysis and comparison.

Background: Simulation-based assessment can complement workplace-based assessment of rare or difficult to assess Entrustable Professional Activities (EPAs). We aimed to compare the use of simulation-based assessment for resuscitation-focused EPAs in three postgraduate medical training programs and describe faculty perceptions of simulation-based assessment. Methods: EPA assessment scores and setting (simulation or workplace) were extracted from 2017-2020 for internal medicine, emergency medicine, and surgical foundations residents at the transition to discipline and foundations of discipline stages. A questionnaire was distributed to clinical competency committee members. Results: Eleven percent of EPA assessments were simulation-based. The proportion of simulation-based assessment did not differ between programs but differed between transition (38%) and foundations (4%) stages within surgical foundations only. Entrustment scores differed between settings in emergency medicine at the transition level only (simulation: 4.82 ± 0.60 workplace: 3.74 ± 0.93). 70% of committee members (n=20) completed the questionnaire. Of those that use simulation-based assessment, 45% interpret them differently than workplace-based assessments. 73% and 100% trust simulation for high-stakes and low-stakes assessment, respectively. Conclusions: The proportion of simulation-based assessment for resuscitation focused EPAs did not differ between three postgraduate medical training programs. Interpretation of simulation-based assessment data between committee members was inconsistent. All respondents trust simulation-based assessment for low-stakes, and the majority for high-stakes assessment. These findings have practical implications for the integration simulation into programs of assessment.

Contexte: Pour les activités professionnelles confiables (APC) qui sont rarement observées ou difficiles à évaluer, une évaluation en séance de simulation peut compléter celle en milieu de travail. Nous avons comparé le recours à une évaluation en séance de simulation pour les APC axées sur la réanimation dans trois programmes de formation médicale postdoctorale et décrit les perceptions de membres du corps professoral à propos de cette modalité d'évaluation. Méthodes: Nous avons extrait les scores et le cadre (simulation ou lieu de travail) d'évaluation des APC de 2017 à 2020 pour les résidents en médecine interne, en médecine d'urgence et en fondements chirurgicaux aux étapes de transition vers la discipline et de fondements de la discipline. Un questionnaire a été distribué aux membres des comités des compétences cliniques. Résultats: Onze pour cent des évaluations d'APC étaient faites lors de séances de simulation. Cette proportion était la même pour tous les programmes, mais dans le cadre des fondements chirurgicaux elle était différente selon qu'il s'agissait de l'étape de transition (38 %) ou de celle des fondements (4 %). Les scores de confiance différaient selon le cadre de l'évaluation uniquement pour les résidents en médecine d'urgence à l'étape de la transition (simulation : 4,82 ± 0,60; lieu de travail : 3,74 ± 0,93). Le questionnaire a été rempli par 70 % des membres des comités (n=20). Parmi ceux qui avaient eu recours à une évaluation en séance de simulation, 45 % avaient interprété les données de l'évaluation différemment de la façon dont ils interprètent les données d'évaluation en milieu de travail. Soixante-treize pour cent et 100 % d'entre eux font confiance à la simulation pour les évaluations à enjeux élevés et à faibles enjeux, respectivement. Conclusions: La proportion d'évaluations en séance de simulation pour les APC axées sur la réanimation était la même dans trois programmes de formation médicale postdoctorale. Les membres des comités de compétences cliniques n'ont pas interprété les données de ce type d'évaluation de manière uniforme. Tous les répondants font confiance à l'évaluation en séance de simulation pour les évaluations à faibles enjeux, et la plupart d'entre eux pour les évaluations à enjeux élevés. Ces données ont des implications pratiques pour l'intégration de la simulation dans les programmes d'évaluation.

Assessment-Seeking Strategies: Navigating the Decision to Initiate Workplace-Based Assessment.

Phenomenon: Competency-based medical education (CBME) relies on workplace-based assessment (WBA) to generate formative feedback (assessment for learning-AfL) and make inferences about competence (assessment of learning-AoL). When approaches to CBME rely on residents to initiate WBA, learners experience tension between seeking WBA for learning and for establishing competence. How learners resolve this tension may lead to unintended consequences for both AfL and AoL. We sought to explore the factors that impact both decisions to seek and not to seek WBA and use the findings to build a model of assessment-seeking strategy used by residents. In building this model we consider how the link between WBA and promotion or progression within a program impacts an individual's assessment-seeking strategy. Approach: We conducted 20 semi-structured interviews with internal medicine residents at Queen's University about the factors that influence their decision to seek or avoid WBA. Using grounded theory methodology, we applied a constant comparative analysis to collect data iteratively and identify themes. A conceptual model was developed to describe the interaction of factors impacting the decision to seek and initiate WBA. Findings: Participants identified two main motivations when deciding to seek assessments: the need to fulfill program requirements and the desire to receive feedback for learning. Analysis suggested that these motivations are often at odds with each other. Participants also described several moderating factors that impact the decision to initiate assessments, irrespective of the primary underlying motivation. These included resident performance, assessor factors, training program expectations, and clinical context. A conceptual framework was developed to describe the factors that lead to strategic assessment-seeking behaviors. Insights: Faced with the dual purpose of WBA in CBME, resident behavior in initiating assessment is guided by specific assessment-seeking strategies. Strategies reflect individual underlying motivations, influenced by four moderating factors. These findings have broad implications for programmatic assessment in a CBME context including validity considerations for assessment data used in summative decision-making including readiness for unsupervised practice.

How workplace-based assessments guide learning in postgraduate education: A scoping review.

INTRODUCTION: Competency-based medical education (CBME) led to the widespread adoption of workplace-based assessment (WBA) with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA which undermine learning goals. Models of workplace-based learning (WBL) provide insight, however, these models excluded WBA. This scoping review synthesizes the primary literature addressing the role of WBA to guide learning in postgraduate medical education, with the goal of identifying gaps to address in future studies. METHODS: The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases, articles up to September 2020 were included. Titles and abstracts were screened by two reviewers, followed by a full text review. Two members independently extracted and analysed quantitative and qualitative data using a descriptive-analytic technique rooted in Billett's four premises of WBL. Themes were synthesized and discussed until consensus. RESULTS: All 33 papers focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and 12 studies (36%) made explicit reference to theory. Aligning with Billett's first premise, results reinforce that learning always occurs in the workplace. WBA helped guide learning goals and enhanced feedback frequency and specificity. Billett's remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. As individuals engage in both work and WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA. Finally, individuals will have different goals, and learn different things, from the same experience. CONCLUSION: Analysing WBA literature through the lens of WBL theory allows us to reframe previously described tensions. We propose that future studies attend to learning theory, and demonstrate alignment with philosophical position, to advance our understanding of assessment-for-learning in the workplace.

Training of Non-expert Users Using Remotely Delivered, Point-of-Care Tele-Ultrasound: A Proof-of-Concept Study in 2 Canadian Communities.

ABSTRACT: Many physicians, particularly those practicing in remote regions, lack training opportunities to develop point-of-care ultrasound (POCUS) skills. This pretest-posttest study quantified the skill improvement of learners after participating in a virtual training program that used real-time, remotely delivered point-of-care tele-ultrasound (tele-POCUS) for teaching and learner feedback provision. Ten physicians practicing in an urban tertiary (Kingston, Ontario, Canada, n = 6) or remote care center (Moose Factory, Ontario, Canada, n = 4) completed a 3-week educational program that consisted of e-learning module review, independent image acquisition practice, and expert-guided tele-POCUS consultations. Pretraining and posttraining assessments were performed to evaluate skill enhancement in image acquisition, image quality, and image interpretation for cardiac and lung/pleura POCUS using a 5-point Likert scale. A total of 76 tele-POCUS consultations were performed during the study period. Significant improvements in image quality were noted following remotely delivered mentorship and guidance (all P < 0.01). In cardiac POCUS, pretraining and posttraining comparisons noted significant improvements in image acquisition (means, 2.69-4.33; P < 0.02), quality (means, 2.40-4.03; P < 0.01), and interpretation (means, 2.50-4.40; P < 0.02). In lung/pleura POCUS, significant improvements in image acquisition (means, 3.00-4.43; P < 0.01), quality (means, 3.23-4.37; P < 0.01), and interpretation (means, 3.00-4.40; P < 0.01) were demonstrated. Introductory ultrasound can be taught to novice users using a virtual, live-streamed training format with tele-POCUS while demonstrating significant enhancement in imaging skills.

Transition to practice curriculum for general internal medicine physicians: scoping review and Canadian national survey.

BACKGROUND: There remains a paucity of evidence for curricula for the transition to practice (TTP) stage of Competence by Design internal medicine (IM) training programs. Current entrustable professional activities are based on expert consensus rather than robust subspecialty-specific needs assessment. METHODS: A scoping review was completed to identify studies with TTP focus. A national survey was conducted to identify transition experiences for general internal medicine physicians. Results were assessed by grounded theory analysis to identify core topics for TTP curricula. RESULTS: Neither scoping review nor national survey identified TTP topics related to the CanMEDS Medical Expert role. Scoping Review: 41 relevant studies were identified. Most (97.6%) were from North America. The most common study types were observational (survey) or curriculum (13/41 31.7% for each). Only two studies were exclusively in IM, and the most common subspecialty studied was surgical (13/41, 31.7%). The most common TTP topics were mentorship, billing and coding, practice management, negotiating contract and job, and financial aspects of practice. National Survey: There were 44 respondents, with the majority (25/44, 56.8%) having completed an IM subspecialty fellowship. Most (38/44) completed medical school in Canada, and most were from academic practice settings (33/44, 75.0%). The most common TTP topics were billing and coding, personal financial planning, practice management, work-life balance and mentorship. Grounded Theory Analysis: There were six themes that encompassed all TTP topics from the scoping review and national survey, being (i) building a career, (ii) continuing professional development, (iii) expectations of the profession, (iv) practice management, (v) Life, health and well-being and (vi) clinical skills. Curriculum competencies and resources for curriculum development were provided. CONCLUSIONS: This study identifies topics critical for curricula development for IM transition to practice. Further research is required to evaluate effectiveness of curricula including topics and themes developed from this scoping review and national survey.

Supervising the senior medical resident: Entrusting the role, supporting the tasks.

INTRODUCTION: Postgraduate competency-based medical education has been implemented with programmatic assessment that relies on entrustment-based ratings. Yet, in less procedurally oriented specialties such as internal medicine, the relationship between entrustment and supervision remains unclear. We undertook the current study to address how internal medicine supervisors conceptualise entrusting senior medical residents while supervising them on the acute care wards. METHODS: Guided by constructivist grounded theory, we interviewed 19 physicians who regularly supervised senior internal medicine residents on inpatient wards at three Canadian universities. We developed a theoretical model through iterative cycles of data collection and analysis using a constant comparative process. RESULTS: On the internal medicine ward, the senior resident role is viewed as a fundamentally managerial and rudimentary version of the supervisor's role. Supervisors come to trust their residents in the senior role through an early 'hands-on' period of assessment followed by a gradual withdrawal of support to promote independence. When considering entrustment, supervisors focused on entrusting a particular scope of the senior resident role as opposed to entrustment of individual tasks. Irrespective of the scope of the role that was entrusted, supervisors at times stepped in and stepped back to support specific tasks. CONCLUSION: Supervisors' stepping in and stepping back to support individual tasks on the acute care ward has an inconsistent relationship to their entrustment of the resident with a particular scope of the senior resident role. In this context, entrustment-based assessment would need to capture more of the holistic perspective of the supervisor's entrustment of the senior resident role. Understanding the dance of supervision, from relatively static overall support of the resident in their role, to fluidly stepping in and out for specific patient care tasks, allows us insight into the affordances of the supervisory relationship and how it may be leveraged for assessment.

The logic behind entrustable professional activity frameworks: A scoping review of the literature.

INTRODUCTION: Entrustable professional activities (EPAs), discrete profession-specific tasks requiring integration of multiple competencies, are increasingly used to help define and inform curricula of specialty training programmes. Although guidelines exist to help guide the developmental process, deciding what logic to use to draft a preliminary EPA framework poses a crucial but often difficult first step. The logic of an EPA framework can be defined as the perspective used by its developers to break down the practice of a profession into units of professional work. This study aimed to map dominant logics and their rationales across postgraduate medical education and fellowship programmes. METHODS: A scoping review using systematic searches within five electronic databases (Medline, Embase, Google Scholar, Scopus and Web of Science) was performed. Dominant logics of included papers were identified using inductive coding and iterative analysis. RESULTS: In total, 42 studies were included. Most studies were conducted in the United States (n = 22; 52%), Canada (n = 6; 14%) and the Netherlands (n = 4; 10%). Across the reported range of specialties, family medicine (n = 4; 10%), internal medicine (n = 4; 10%), paediatrics (n = 3; 7%) and psychiatry (n = 3; 7%) were the most common. Three dominant logics could be identified, namely, 'service provision', 'procedures' and/or 'disease or patient categories'. The majority of papers (n = 37; 88%) used two or more logics when developing EPA frameworks (median = 3, range = 1-4). Disease or patient groups and service provision were the most common logics used (39% and 37%, respectively). CONCLUSIONS: Most programmes used a combination of logics when trying to capture the essential tasks of a profession in EPAs. For each of the three dominant logics, the authors arrived at a definition and identified benefits, limitations and examples. These findings may potentially inform best practice guidelines for EPA development.

Trainee Uncertainty around Intervening When Patients Decompensate.

BACKGROUND: Trainees in acute care specialties often grapple with the decision to perform an invasive procedure in a rapidly decompensating patient, for whom the benefits and risks are inherently uncertain. The difference between trainees who know when to act and when to seek supervision and those who do not is often linked to individual trainee psychological and cultural perceptions of uncertainty. But how much comfort with uncertainty relates to the situational context rather than the trainee traits is underexplored. OBJECTIVE: The objective of this study was to explore trainee actions around decompensating patients and assess the degree to which invasive intervention and supervision seeking depend on situational certainty or individual trait-based perceptions of uncertainty. METHODS: A total of 41 internal medicine residents completed a survey to measure anxiety related to uncertainty using the Physicians' Reactions to Uncertainty (PRU) tool and to measure uncertainty avoidance using the Values Survey Module (VSM) before responding to 14 written emergency situations. Half of the scenarios contain sufficient diagnostic certainty to warrant aggressive intervention, and half lack sufficient diagnostic clarity to offset the risk of intervention. Mixed multivariable modeling was used to identify the relationship between planned invasive intervention, situational uncertainty, and trait-based perceptions of uncertainty measured in the PRU and VSM. RESULTS: Trainees' first actions were appropriate in 60% of cases. Multivariable modeling suggested that situational certainty was more predictive of upfront intervention (odds ratio [OR], 30.5; P < 0.0001) than trait-based PRU (OR, 1.22; P = 0.05) and VSM (OR, 1.73; P < 0.0001). Similarly, situational certainty was more predictive of reduced supervision seeking (OR, 0.20; P < 0.0001) than trait-based PRU (OR, 2.03; P < 0.001) and VSM (P = not significant). CONCLUSIONS: Situation-specific certainty was more strongly correlated with invasive intervention in cases of decompensated patients than individual trainee traits. Focusing on trainee contextual understanding of procedural risk-benefit ratios in decompensating patients holds more promise for improving trainee actions and supervision seeking than tackling their perceptions around uncertainty.

Resident and attending perceptions of direct observation in internal medicine: a qualitative study.

OBJECTIVES: Direct observation is the foundation of assessment and learning in competency-based medical education (CBME). Despite its importance, there is significant uncertainty about how to effectively implement frequent and high-quality direct observation. This is particularly true in specialties where observation of non-procedural skills is highly valued and presents unique challenges. It is therefore important to understand perceptions of direct observation to ensure successful acceptance and implementation. In this study, we explored perceptions of direct observation in internal medicine. METHODS: We interviewed internal medicine attending physicians (n = 9) and residents (n = 8) at the University of Toronto, purposively sampled for diversity. Using a constructivist grounded theory approach, constant comparative analysis was performed to develop a framework to understand perceptions of direct observation on the clinical teaching units. RESULTS: Participants articulated a narrow perception of what constitutes direct observation, in contrast to their own descriptions of skills that were observed. This resulted in the perception that certain valuable skills that participants felt were routinely observed were nonetheless not 'directly observable', such as clinical reasoning, observed through case presentations and patient care discussions. Differentiating direct observation from informal observation led to overestimation of the time and resource requirements needed to enhance direct observation, which contributed to scepticism and lack of engagement related to CBME implementation. CONCLUSIONS: In an internal medicine training programme, perceptions of what constitutes direct observation can lead to under-recognition and hinder acceptance in workplace-based assessment and learning. Our results suggest a reframing of 'direct observation' for residents and attending physicians, by explicitly identifying desired skills in non-procedurally-based specialties. These findings may help CBME-based training programmes improve the process of direct observation, leading to enhanced assessment and learning.

Deliberate practice as a framework for evaluating feedback in residency training.

OBJECTIVE: Using the theory of deliberate practice, a key component of Ericsson's theory of expertise development, this study aims to evaluate the quality of written feedback given to learners. METHODS: The authors created a feedback scoring system based on the key elements of deliberate practice and used it to assess the quality of written feedback provided to residents in 205 mini-CEX encounter forms. Scores were assigned to each feedback entry for identification of the following: Task, performance gap and action plan. RESULTS: The scoring system allowed for reliable identification of the components that facilitate deliberate practice in written feedback provided to trainees. However, only one of these components was identified in 70% of the feedback entries. A specific task was identified in 56%, whereas specific performance gaps and action plans were identified in only 3.9% and 13.7% of encounters, respectively. CONCLUSIONS: Scoring written feedback identified that tasks were often specifically described, but performance gaps and action plans were less frequently and specifically mentioned. Educators might improve feedback effectiveness by better articulating to trainees the gap between their performance and an expert standard, as well as by providing them with specific learning plans.

Establishing a swine model to compare vascular prostheses in a contaminated field.

OBJECTIVE: Patch arteriotomies are performed during many vascular procedures. Whereas synthetic materials are generally felt to be inappropriate for infected environments, the suitability of glutaraldehyde-treated bovine pericardium (GBP), a biologic material, in such instances is unknown. Our main objectives were to develop an animal model to study vascular prostheses while comparing the infectability of polyester (Dacron) and GBP in a topically infected environment. METHODS: Twenty-three pigs underwent transabdominal patch arteriotomy of the infrarenal aorta with either Dacron or GBP. The patches were inoculated with sterile saline (1 per group), Staphylococcus aureus 10(4) colony-forming units (CFUs) (4 per group), or S. aureus 10(5) CFUs (6 per group). At 3 wk, the animals were euthanized, and the patches were removed via a left retroperitoneal approach. Specimens were collected for microbiologic and histologic analysis. RESULTS: One animal from each group inoculated with 10(5) CFUs died during the study period, and another died immediately postoperatively of an airway complication. All aortas were patent and without evidence of pseudoaneurysm formation. Gross abscesses were found in 4/6 Dacron and 5/6 GBP animals receiving 10(5) CFUs. Similarly, 4/6 animals implanted with Dacron and 5/6 animals implanted with GBP had positive tissue cultures. A histologic grading system of inflammation substantiated the culture results. CONCLUSIONS: No significant difference exists between Dacron and GBP to resist bacterial infection at 3 wk. We have established a reproducible in vivo model to study arterial patch materials in a topically infected environment.

Metalworking fluids: oil mist and beyond.

This article is based upon my own experiences with metalworking fluids and the adverse health effects and medical conditions associated with exposure to metalworking fluids. I have researched and witnessed the benefits that can be achieved when metalworking fluids are properly maintained and managed. My experiences have provided insight into how a shop operates, including comprehension of the equipment used, processes, mist generating points, engineering controls currently being adopted, and procedures that are used to maintain metalworking fluids. I have been able to share my personal experiences with the country's leading experts in the field of metalworking fluids. I have presented my insights on the topic in Washington, D.C., to the Standard Advisory Committee of OSHA, as well as at many other conferences nationwide. I have provided awareness training for a number of union and nonunion workers. Being a part of developing successful metal removal fluid programs, I realize the importance of transferring and sharing information. Many times an organization is not fully aware of certain conditions and how to combat them. My mission and intent is to properly educate those who are exposed to the harm that metalworking fluids can invoke and to inform those involved of the possible methods of reducing long- and short-term risk. One thing that must be kept in mind is the way we view these fluids. Many shops categorize the fluids as a type of "operating expense" when they should actually be seen as a sort of investment. Just as performing a scheduled maintenance on a machine promises the best possible longevity of that machine, the upkeep of metalworking fluid also provides longer "tool life." Monitoring and maintaining the fluids also provides for more effective and efficient productivity. If we fail to consider that proper management of the fluids can cut cost dramatically, then we will miss out on the financial impact they can have on a company. Try looking at the fluids as a liquid tool. Doing so I believe will bring a better understanding of the value of a successful metalworking fluids program. With this new understanding, it can be seen just who must play a role in the management of metalworking fluids. The employees who deal with the daily tasks involving the coolant play a major part. They are on the floor where these metalworking fluids are being used. In many shops, it is assumed that the environmental health & safety departments are responsible for standard operating procedures and management of fluids. The EH&S department should only be responsible for the protection from exposure and the transfer of information regarding policy and procedure to their employees. Not all shops have the resources required to develop and implement the proper standard operating procedure. Therefore, we must understand that what is feasible for one may not be for another. Companies that lack the sufficient resources should not be neglected. It is crucial that awareness of proper standard operating procedure is shared with everyone involved with the fluids in order to provide proper metalworking fluids management. Fluids are as dynamic as the formulations themselves (complex & dynamic). These fluids can quickly become contaminated with foreign materials and chemicals, thereby become aerosolized into mist. With proper education and training, one will be able to control what gets aerosolized.