Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias in feedback.

The entrustment framework redirects assessment from considering only trainees' competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic characteristics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differentially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors' themes predominantly reflected skills related to patient presentations, while trainees' themes were broader-including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment, p < 0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1-4 scale, p < 0.05). We also found biases tied to demographic characteristics: trainees' documentation reflected more positive sentiment in the case of male trainees (+ 1.3%, p < 0.05) and of trainees underrepresented in medicine (UIM) (+ 1.3%, p < 0.05). Entrustment ratings did not appear to reflect these biases, neither when documented by trainee nor supervisor. As such, bias appeared to influence the emotive language trainees used to document entrustment more than the degree of entrustment they experienced. Mitigating these biases is nonetheless important because they may affect trainees' assimilation into their roles and formation of trusting relationships.

Exploring Interprofessional Development of Entrustable Professional Activities For Pediatric Intensive Care Fellows: A Proof-of-Concept Study.

Phenomenon: Entrustable professional activities (EPAs) delineate major professional activities that an individual in a given specialty must be "entrusted" to perform, ultimately without supervision, to provide quality patient care. Until now, most EPA frameworks have been developed by professionals within the same specialty. As safe, effective, and sustainable health care ultimately depends on interprofessional collaboration, we hypothesized that members of interprofessional teams might have clear and possibly additional insight into which activities are essential to the professional work of a medical specialist. Approach: We recently employed a national modified Delphi study to develop and validate a set of EPAs for Dutch pediatric intensive care fellows. In this proof-of-concept study, we explored what pediatric intensive care physicians' non-physician team members (physician assistants, nurse practitioners, and nurses) constitute as essential professional activities for PICU physicians and how they regarded the newly developed set of nine EPAs. We compared their judgments with the PICU physicians' opinions. Findings: This study shows that non-physician team members share a mental model with physicians about which EPAs are indispensable for pediatric intensive care physicians. Despite this agreement however, descriptions of EPAs are not always clear for non-physician team members who have to work with them on a daily basis. Insights: Ambiguity as to what an EPA entails when qualifying a trainee can have implications for patient safety and trainees themselves. Input from non-physician team members may add to the clarity of EPA descriptions. This finding supports the involvement of non-physician team members in the developmental process of EPAs for (sub)specialty training programs.

Medical competence as a multilayered construct.

BACKGROUND: The conceptualisation of medical competence is central to its use in competency-based medical education. Calls for 'fixed standards' with 'flexible pathways', recommended in recent reports, require competence to be well defined. Making competence explicit and measurable has, however, been difficult, in part due to a tension between the need for standardisation and the acknowledgment that medical professionals must also be valued as unique individuals. To address these conflicting demands, a multilayered conceptualisation of competence is proposed, with implications for the definition of standards and approaches to assessment. THE MODEL: Three layers are elaborated. This first is a core layer of canonical knowledge and skill, 'that, which every professional should possess', independent of the context of practice. The second layer is context-dependent knowledge, skill, and attitude, visible through practice in health care. The third layer of personalised competence includes personal skills, interests, habits and convictions, integrated with one's personality. This layer, discussed with reference to Vygotsky's concept of Perezhivanie, cognitive load theory, self-determination theory and Maslow's 'self-actualisation', may be regarded as the art of medicine. We propose that fully matured professional competence requires all three layers, but that the assessment of each layer is different. IMPLICATIONS: The assessment of canonical knowledge and skills (Layer 1) can be approached with classical psychometric conditions, that is, similar tests, circumstances and criteria for all. Context-dependent medical competence (Layer 2) must be assessed differently, because conditions of assessment across candidates cannot be standardised. Here, multiple sources of information must be merged and intersubjective expert agreement should ground decisions about progression and level of clinical autonomy of trainees. Competence as the art of medicine (Layer 3) cannot be standardised and should not be assessed with the purpose of permission to practice. The pursuit of personal excellence in this level, however, can be recognised and rewarded.

Do summative entrustment decisions actually lead to entrustment?

BACKGROUND: Entrustable professional activities (EPAs) were introduced across Dutch postgraduate programmes between 2017 and 2019. We aimed to understand the extent to which residents actually were granted increased clinical responsibility upon receiving summative entrustment for an EPA, a critical feature of its use. METHODS: A survey study was conducted among all Dutch residents who started dermatology training in 2018 and 2019 and all Dutch dermatology programme directors (PDs). We chose an EPA designed for early entrustment in residency (identification, treatment and care regarding a simple dermatological problem in the ambulatory setting). The survey contained two hypothetical clinical cases that aligned with this EPA. The questions were aimed to determine whether and when residents should request supervision. Similar questions were posed to PDs. FINDINGS: Twenty four residents (56%) and 19 PDs (79%) completed the survey. The majority of the residents (65%) and PDs (63%) confirmed that competent dermatology residents (level 4) are generally allowed to perform EPA1 unsupervised, particularly when seeing patients from GPs. However, still a substantial proportion of the level 4 residents, working in University Medical Centers (36%) indicated that they had to request supervision in the assessment of these patients. For 2nd opinions, the results were typically the opposite. DISCUSSION AND CONCLUSION: This study demonstrated that, at least in one specialty and one country, the introduction of EPAs and entrustment decision making procedure generally led to the intended autonomy of the resident.

Mentorship in health professions education - an AMEE guide for mentors and mentees: AMEE Guide No. 167.

This AMEE guide discusses theoretical principles and practical strategies for health professions educators to promote impactful mentoring relationships. Traditional definitions are challenged, distinctions are made between roles such as mentor, advisor, coach and sponsor. As educational environments change and options for professional development expand, we argue that the traditional dyadic format of mentoring alone will not help mentees to maximise their professional growth. Newer formats of mentoring are discussed in detail and their advantages and disadvantages compared. We use a variety of theoretical concepts to anchor the practice of mentorship: self-focussed and other-focussed motives; psychological safety; personal interpretive framework; Daloz model for balancing support and challenge; zone of proximal development; communities of practice; and development along multiple layers of competence. Recommended strategies for effective mentoring are based on extensive review of literature, as well as combined professional mentoring experiences of the authors. We use key principles from the theories described and phases of mentoring relationships as foundations for the suggested best practices of mentorship. Finally, we emphasise the role of mentees in their own professional development and provide tips for them on seeking mentors, expanding their mentoring network and taking the lead in setting the agenda during mentoring meetings and formulating action plans for their own advancement.

Facilitating flexibility in postgraduate nursing education through entrustable professional activities to address nursing shortages and career prospects.

AIM: To communicate how the Dutch postgraduate nursing education landscape was redesigned using entrustable professional activities (EPAs). The goal of this initiative was to reduce training time, enhance transfer across nursing specialities and contribute to a better-aligned continuum of initial education, postgraduate education and continuing professional development. BACKGROUND: Nursing shortages continue to worsen worldwide. An approach to address this growing shortage is to create a more flexible postgraduate training structure, offering training in the just-in-time and just-as-needed models. EPAs can be used as building blocks for training and assessment. Experience with EPAs (i.e. units of professional practice that can be entrusted once a trainee has demonstrated the required competencies) in health professions education, including nursing, is rising rapidly. While EPAs are largely used to create training flexibility within a programme, they can also be used to create flexibility across programmes. In 2018-2022, training hospitals and education institutions in the Netherlands collaborated in the CZO Flex Level Project to redesign the postgraduate nursing education landscape using EPAs. DISCUSSION: The implementation of a flexible postgraduate nursing education model nationwide will face several challenges. An overview of these challenges and suggestions for future research on the effects of the new structure on nursing competence, satisfaction and career development are provided. CONCLUSION: EPAs can imbue flexibility within and across training programmes. Designing an EPA-based educational landscape requires nationally coordinated efforts. IMPLICATIONS FOR NURSING POLICY: Redesigning educational structures to allow for more flexibility is critical to address major societal challenges in healthcare.

Clinical supervision under pressure: a qualitative study amongst health care professionals working on the ICU during COVID-19.

PURPOSE: The unprecedented influx of patients in 2020 with COVID-19 to intensive care units (ICU) required redeployment of healthcare professionals without adequate previous ICU-training. In these extraordinary circumstances, pivotal elements of effective clinical supervision emerged. This study sets out to explore the nature, aspects and key features of supervision under highly demanding circumstances among certified and redeployed health-care professionals on COVID-19 ICUs. MATERIALS AND METHODS: A prospective qualitative, single center, semi-structured interview study among healthcare professionals at COVID-19 ICUs at University Medical Center Utrecht, the Netherlands between July and December 2020. Interview data were analyzed using an inductive coding style. RESULTS: A total of 13 certified and 13 redeployed health'hcare professionals, including physicians, nurses, and operation room technicians participated. Seven themes were identified as essential for both certified (supervisors) and redeployed (trainees) personnel: an open attitude, observing boundaries, gauging coworkers' capacities, being available, providing feedback, continuity in care and teams, and combining supervision with workload. CONCLUSIONS: This study provides seven recommendations for both supervisors and trainees to help optimize clinical supervision. They align with the known five factors determining entrustment and supervision (trainee, supervisor, task, context, and relationship). To ensure good clinical supervision, be it either during normal circumstances or under pressure, efforts should primarily focus on factors that are within a supervisor or trainee's span of control. MESH: Clinical supervision, interprofessional, COVID-19, Intensive Care.

Would I Trust or Will I Trust? The Gap between Entrustment Determinations and Entrustment Decisions for Trainees in Pharmacy and Other Health Professions.

Entrustable Professional Activities (EPAs) and entrustment decision making are rapidly becoming mainstream in competency-based education in the health professions. EPAs are the units of professional practice to entrust graduates with once they have developed the required competencies. They were conceived to enable a gradual increase in professional autonomy during training, by allowing trainees to practice activities in which they have demonstrated they have mastered well, with decreasing supervision. However, practicing health care unsupervised generally requires licensure. The question for pharmacy education, as well as for undergraduate medical education, is can students be given any autonomy in practice, even when they have fully mastered an EPA yet remain unlicensed? While entrustment decisions for licensed practitioners have autonomy consequences, some educators in undergraduate programs speak of 'entrustment determinations', to avoid decisions about students that affect patient care, in other words saying, we would trust you, rather than we will trust you. However, graduating learners without the experience of responsibility and reasonable autonomy creates a gap with full practice responsibilities, which may jeopardize patient safety after training. What can programs do to retain the power of using EPAs while at the same time guarding patient safety?

Twelve tips to develop entrustable professional activities.

Entrustable professional activities (EPAs), units of professional practice that require proficient integration of multiple competencies and can be entrusted to a sufficiently competent learner, are increasingly being used to define and inform curricula of health care professionals. The process of developing EPAs can be challenging and requires a deep yet pragmatic understanding of the concepts underlying EPA construction. Based on recent literature and the authors' lessons learned, this article provides the following practical and more or less sequential recommendations for developing EPAs: [1] Assemble a core team; [2] Build up expertise; [3] Establish a shared understanding of the purpose of EPAs; [4] Draft preliminary EPAs; [5] Elaborate EPAs; [6] Adopt a framework of supervision; [7] Perform a structured quality check; [8] Use a Delphi approach for refinement and/or consensus; [9] Pilot test EPAs; [10] Attune EPAs to their feasibility in assessment; [11] Map EPAs to existing curriculum; [12] Build a revision plan.

Preceptors' considerations when entrusting professional activities to postgraduate nursing students: A qualitative study.

BACKGROUND: Nurse-preceptors regularly struggle to evaluate students' readiness to take care of patients unsupervised, even with sophisticated workplace-based assessment tools. Preceptors' gut feelings are not always captured well, but are critical for judgement of readiness for learner entrustment with care tasks. Studies in medical education report features that clinicians consider important when trusting students with clinical responsibilities that might also apply in nursing. OBJECTIVES: To unravel preceptors' considerations when entrusting professional activities to postgraduate nursing students. The findings may contribute to the improvement of workplace-based assessments and the training of preceptors. METHODS: Thematic analysis of semi-structured interviews with sixteen nurse-preceptors from three postgraduate nursing specialisations in Dutch hospitals. RESULTS: Three themes emerged: CONCLUSIONS: For preceptors of postgraduate nursing students, entrustment requires more than merely insight into objectively measurable competencies. Entrusting is accompanied by subjectivity related to what preceptors expected of students. These expectations are in line with suggested factors in the literature-capability, integrity, reliability, agency, and humility-considered before entrusting students with clinical responsibilities identified in medical training. Entrusting is also accompanied by what preceptors realise about their own role in entrustment decisions. Combining different information sources made assessment more transparent and the implicit more explicit.

Harmen Tiddens (1923-2002): A butterfly's impact on medical education.

The history of medical education scholarship is often overlooked. Moreover, although productive educational researchers are often recognized for their scholarly contributions in peer-reviewed publications, the impact of others who are not active medical education researchers may be unknown to contemporary educators and scholars. This short paper describes the contributions of Harmen Tiddens, MD PhD to Utrecht and Maastricht Universities where he established the environment and supports for scholars to undertake profoundly influential work. Through his leadership as the founding Dean of Maastricht's new medical school, Dr. Tiddens facilitated educational principles that became exemplary for generations of health professions education scholars and curriculum developers, nationally and internationally.

Entrustable professional activities in postgraduate general surgery training: a scoping review protocol.

INTRODUCTION: The competency-based medical education (CBME) movement continues to gain momentum in postgraduate physician training, resulting in increasing interest among surgical training programmes on how to implement it effectively. Entrustable professional activities (EPAs) were introduced to connect competencies (characteristics/abilities of learners) and the professional activities to be entrusted to them on qualification/graduation. Although reviews related to the field of general surgery have been published on specific aspects of CBME, for example, workplace-based assessment, there is a paucity of published guidance available for surgeon-educators to plan and implement CBME using an EPA framework. This scoping review aims to provide an overview of the current use of EPAs in general surgery. METHODS AND ANALYSIS: This scoping review will be based on the original five-stage approach described by Arksey and O'Malley with the addition of protocol-specific items from the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols extension for scoping reviews. The primary research question focuses on describing how EPAs have been used in general surgery and where potential gaps remain that warrant further study. The review will include peer-reviewed journal publications and an extensive review of grey literature sources. The following databases will be searched for published literature from 1 January 2005 to 31 December 2022: PubMed; CINAHL, Africa-Wide Information, PsycInfo and ERIC (via EBSCOhost); Scopus; and SciELO (via Web of Science). Studies that describe the use of EPAs for curriculum design, teaching and/or assessment of competence in postgraduate general surgery training will be eligible for inclusion. ETHICS AND DISSEMINATION: The institutional ethics board of the University of Cape Town has granted a waiver of formal approval requirement. The dissemination strategy includes publication of results in peer-reviewed journals, presentation at international conferences and presentation to relevant stakeholders as deemed appropriate.

The Fate of Entrustable Professional Activities After Graduation: A Survey Study Among Graduated Physician Assistants.

INTRODUCTION: Entrustable professional activities (EPAs) are used during training but may also have significance after graduation. This domain has not yet been much explored. We investigated the use of EPAs among alumni of Master Physician Assistant programs in the Netherlands, where EPAs have been used since 2010. We aim to describe if and how EPAs have a role after graduation. Are EPAs used for continuous entrustment decisions or when shifting work settings? METHODS: In September 2020, all registered Dutch physician assistants (PAs) ( n = 1441) were invited to participate in a voluntary, anonymous survey focusing on the use of EPAs after graduation. Survey items included questions regarding the use and usefulness of EPAs, views on continuous entrustment decisions, and how PAs document their competence. RESULTS: A total of 421 PAs (response rate 29%) yielded information about the significance of EPAs after training. Among the respondents, 60% ( n = 252) reported adding new competencies and skills after graduation. One-third ( n = 120) of the respondents were trained in EPA-based programs. Almost all EPA-trained PAs (96%; n = 103) considered the EPA structure suitable to maintain and renew entrustment. Furthermore, PAs reported continued use of the EPA framework to seek formal qualifications through entrustment decisions. CONCLUSION: This study shows that EPAs can play an ongoing role for PAs after graduation. EPA-trained PAs overwhelmingly support the continuous use of the EPA framework for entrustment decisions from graduation until retirement.

Transdisciplinary entrustable professional activities.

Healthcare has become highly specialized. Specialists, in medicine as well as in nursing, determine much of the high quality of current health care. But healthcare has also become increasingly fragmented, with professionals trained in separate postgraduate silos, with boundaries often difficult to cross. While a century ago, generalists dominated patient care provision, now specialists prevail and risk becoming alienated from each other, losing the ability to adapt to neighboring professional domains. Current health care requires a flexible workforce, ready to serve in multiple contexts, as the COVID-19 crisis has shown.The new concept of transdisciplinary entrustable professional activities, EPAs applicable in more than one specialty, was recently conceived to enhance collaboration and transfer between educational programs in postgraduate nursing in the Netherlands.In this paper, we reflect on our experiences so far, and on practical and conceptual issues concerning transdisciplinary EPAs, such as: who should define, train, assess, and register transdisciplinary EPAs? How can different prior education prepare for similar EPAs? And how do transdisciplinary EPAs affect professional identity?We believe that transdisciplinary EPAs can contribute to creating more flexible curricula and hence to a more coherent, collaborative healthcare workforce, less determined by the boundaries of traditional specialties.

Mobile technologies to support workplace-based assessment for entrustment decisions: Guidelines for programs and educators: AMEE Guide No. 154.

With the rise of competency-based medical education and workplace-based assessment (WBA) since the turn of the century, much has been written about methods of assessment. Direct observation and other sources of information have become standard in many clinical programs. Entrustable professional activities (EPAs) have also become a central focus of assessment in the clinical workplace. Paper and pencil (one of the earliest mobile technologies!) to document observations have become almost obsolete with the advent of digital technology. Typically, clinical supervisors are asked to document assessment ratings using forms on computers. However, accessing these forms can be cumbersome and is not easily integrated into existing clinical workflows. With a call for more frequent documentation, this practice is hardly sustainable, and mobile technology is quickly becoming indispensable. Documentation of learner performance at the point of care merges WBA with patient care and WBA increasingly uses smartphone applications for this purpose.This AMEE Guide was developed to support institutions and programs who wish to use mobile technology to implement EPA-based assessment and, more generally, any type of workplace-based assessment. It covers backgrounds of WBA, EPAs and entrustment decision-making, provides guidance for choosing or developing mobile technology, discusses challenges and describes best practices.

Investigating feedback orientation in medical learners.

BACKGROUND/PURPOSE: Feedback processes in health professions education (HPE) are not always successful. While recommendations to improve feedback provision dominate the literature, studying specific learner attributes that impact feedback uptake may also improve feedback processes. Feedback orientation is a concept from management science involving four dimensions of learner attributes and attitudes that impact their feedback uptake: utility, accountability, social awareness, and feedback self-efficacy. Feedback orientation may represent a valuable concept in HPE. We aimed to understand medical learners' feedback orientation at different stages in their development. METHODS: We used the Feedback Orientation Scale, a 20-item survey instrument, for a cross-sectional analysis of feedback orientation in medical students and Internal Medicine residents at one large academic center. We performed descriptive statistics and analysis of variance for data analysis. RESULTS: We found the same factors (dimensions) to feedback orientation in our population as in management science. Overall feedback orientation scores were high and were largely consistent across trainee levels. Utility was the domain that was highest across learners, whereas feedback self-efficacy was lowest. CONCLUSIONS: Feedback orientation represents a useful concept to explore medical learners' attitudes toward feedback's role in their development. The four domains can help guide further nuanced feedback research and application.[Box: see text].

Preliminary evidence for a Tool to Observe the Construction of Knowledge in Interprofessional teams (TOCK-IP).

Collaborative knowledge construction (KC) is an important process in interprofessional learning and a logical assessment target. A tool supporting the formative evaluation of KC behaviors ideally would be: 1) applicable to interprofessional teams of learners in clinical contexts; 2) informed by contemporary learning frameworks; 3) feasible and useful. No existing assessment tool meets these criteria. This paper describes the development and preliminary validity evidence for a Tool for Observing Construction of Knowledge in Interprofessional teams (TOCK-IP). Following literature review and needs assessment, the TOCK-IP was drafted based upon Gunawardena's five-phase KC model. Educational expert review established content validity. Response process and internal structure validity, feasibility, and utility were assessed through step-wise evaluation. Faculty raters applied the tool to four videos of simulated interactions between health professions learners. Faculty ratings were compared to expert consensus ratings. Thematic analysis of post-rating survey and debrief allowed assessment of feasibility and utility. Across videos, faculty raters' agreement was fair (n = 25; Fleiss' kappa = 0.40, <0.001). Excellent agreement (95%) was found for raters' scores compared to consensus rating. Faculty supported tool feasibility and utility. The TOCK-IP meets the three criteria for evaluating team-level KC and offers a progression roadmap to help learners move toward collaborative learning.

In pursuit of a better transition to selected residencies: a quasi-experimental evaluation of a final year of medical school dedicated to the acute care domain.

BACKGROUND: Medical schools seek the best curricular designs for the transition to postgraduate education, such as the Dutch elective-based final, 'transitional' year. Most Dutch graduates work a mean of three years as a physician-not-in-training (PNIT) before entering residency training. To ease the transition to selected specialties and to decrease the duration of the PNIT period, UMC Utrecht introduced an optional, thematic variant of the usual transitional year, that enables the development of theme-specific competencies, in addition to physicians' general competencies. METHODS: We introduced an optional transitional year for interested students around the theme of acute care, called the Acute Care Transitional Year (ACTY). This study aimed to evaluate the ACTY by judging whether graduates meet postgraduate acute care expectations, indicating enhanced learning and preparation for practice. In a comprehensive assessment of acute care knowledge, clinical reasoning, skills, and performance in simulations, we collected data from ACTY students, non-ACTY students interested in acute care, and PNITs with approximately six months of acute care experience. RESULTS: ACTY graduates outperformed non-ACTY graduates on skills and simulations, and had higher odds of coming up to the expectations faculty have of a PNIT, as determined by global ratings. PNITs did better on simulations than ACTY graduates. DISCUSSION: ACTY graduates show better resemblance to PNITs than non-ACTY graduates, suggesting better preparation for postgraduate acute care challenges. CONCLUSION: Transitional years, offering multidisciplinary perspectives on a certain theme, can enhance learning and preparedness for entering residency.

Are We Facing the End of Gross Anatomy Teaching as We Have Known It for Centuries?

The status of anatomy education in undergraduate medical education has dramatically changed over the course of the past century. From the most important and time-consuming component of the preclinical program, anatomy education has reduced in size and status, and yielded in curricular space to accommodate other disciplines and topics. Meanwhile, radiology has become more prominent, as a means to visualize anatomy, not only in clinical care but also in education. For this perspective paper, the authors, all with backgrounds in anatomy, radiology and/or medical education, conducted structured conversations with several academic colleagues with similar backgrounds, reviewed pertinent literature and analyzed the causes of the historical decline of a knowledge domain of medical education, that nevertheless is widely considered essential for medical students and graduates. After this analysis, the authors propose four ways forward. These directions include systematic peer teaching and development of anatomy education as a scholarly domain, further vertical integration with postgraduate medical education, full integration with radiology education, and capitalizing on educational technology. Schools in several industrialized countries have made steps in these directions, which can be further strengthened. In less affluent countries, and in countries with curricula strongly determined by tradition, these steps are less easy to make. To respond to changes in global health and health care, combined with the inevitable technological progress, and international mobility, we believe all schools will move in these directions, slower or faster.