The Association of ACGME Milestones With Performance on American Board of Surgery Assessments: A National Investigation of Surgical Trainees.

OBJECTIVE: To determine the relationship between, and predictive utility of, milestone ratings and subsequent American Board of Surgery (ABS) vascular surgery in-training examination (VSITE), vascular qualifying examination (VQE), and vascular certifying examination (VCE) performance in a national cohort of vascular surgery trainees. BACKGROUND: Specialty board certification is an important indicator of physician competence. However, predicting future board certification examination performance during training continues to be challenging. METHODS: This is a national longitudinal cohort study examining relational and predictive associations between Accreditation Council for Graduate Medical Education (ACGME) Milestone ratings and performance on VSITE, VQE, and VCE for all vascular surgery trainees from 2015 to 2021. Predictive associations between milestone ratings and VSITE were conducted using cross-classified random-effects regression. Cross-classified random-effects logistic regression was used to identify predictive associations between milestone ratings and VQE and VCE. RESULTS: Milestone ratings were obtained for all residents and fellows(n=1,118) from 164 programs during the study period (from July 2015 to June 2021), including 145,959 total trainee assessments. Medical knowledge (MK) and patient care (PC) milestone ratings were strongly predictive of VSITE performance across all postgraduate years (PGYs) of training, with MK ratings demonstrating a slightly stronger predictive association overall (MK coefficient 17.26 to 35.76, ß = 0.15 to 0.23). All core competency ratings were predictive of VSITE performance in PGYs 4 and 5. PGY 5 MK was highly predictive of VQE performance [OR 4.73, (95% CI, 3.87-5.78), P <0.001]. PC subcompetencies were also highly predictive of VQE performance in the final year of training [OR 4.14, (95% CI, 3.17-5.41), P <0.001]. All other competencies were also significantly predictive of first-attempt VQE pass with ORs of 1.53 and higher. PGY 4 ICS ratings [OR 4.0, (95% CI, 3.06-5.21), P <0.001] emerged as the strongest predictor of VCE first-attempt pass. Again, all subcompetency ratings remained significant predictors of first-attempt pass on CE with ORs of 1.48 and higher. CONCLUSIONS: ACGME Milestone ratings are highly predictive of future VSITE performance, and first-attempt pass achievement on VQE and VCE in a national cohort of surgical trainees.

Trends in Operative Case Logs of Chief Residents in Surgery by Sex and Race: A 5 - Year National Study.

OBJECTIVES: A recent study of 21 institutions noted significant differences between number of cases reported during general surgery residency by trainees who are Underrepresented in Medicine (URiM) versus trainees who are not Underrepresented in Medicine (non-URiM). This study also identified differences between female residents and male residents. We partnered with the Accreditation Council for Graduate Medical Education to examine case logs reported from all accredited general surgery programs in the United States. This is the first time this data has been examined nationally. METHODS: We examined total case logs submitted by graduating residents between 2017 and 2022. Group differences in mean reported case logs were examined using paired t- tests for female versus male and URiM versus non- URiM overall case numbers. RESULTS: A total of 6,458 residents submitted case logs from 319 accredited programs. Eight-hundred and fifty-four (13%) were URiM and 5,604 (87%) were non-URiM. Over the 5-year study period, URM residents submitted 1096.95 (SD +/- 160.57) major cases versus 1115.96 (+/- 160.53) for non-URiM residents (difference =19 cases, P=0.001). Case logs were submitted by 3,833 (60.1%) male residents and 2,625 (39.9%) female residents over the five-year study period. Male residents reported 1128.56 (SD +/- 168.32) cases versus 1091.38 (+/- 145.98) cases reported by females (difference=37.18, P<0.001). When looking at Surgeon Chief and Teaching Assistant cases, there was no significant difference noted between cases submitted by URiM versus non- URiM residents. However, male residents reported significantly more in both categories than their female peers (P<0.001). CONCLUSIONS: Overall, URiM residents submitted fewer cases in the five- year study period than their non-URiM peers. The gap in submitted cases between male and female residents was more pronounced, with male residents submitting significantly more cases than their female counterparts. This finding was consistent and statistically significant throughout the entire study period, in most case categories, and without narrowing of difference over time. A difference of 30-40 cases can amount to 1-3 months of surgical training and is a concerning national trend deserving the attention of every training program and our governing institutions.

Gender Differences in Work-Based Assessment Scores and Narrative Comments After Direct Observation.

BACKGROUND: While some prior studies of work-based assessment (WBA) numeric ratings have not shown gender differences, they have been unable to account for the true performance of the resident or explore narrative differences by gender. OBJECTIVE: To explore gender differences in WBA ratings as well as narrative comments (when scripted performance was known). DESIGN: Secondary analysis of WBAs obtained from a randomized controlled trial of a longitudinal rater training intervention in 2018-2019. Participating faculty (n = 77) observed standardized resident-patient encounters and subsequently completed rater assessment forms (RAFs). SUBJECTS: Participating faculty in longitudinal rater training. MAIN MEASURES: Gender differences in mean entrustment ratings (4-point scale) were assessed with multivariable regression (adjusted for scripted performance, rater and resident demographics, and the interaction between study arm and time period [pre- versus post-intervention]). Using pre-specified natural language processing categories (masculine, feminine, agentic, and communal words), multivariable linear regression was used to determine associations of word use in the narrative comments with resident gender, race, and skill level, faculty demographics, and interaction between the study arm and the time period (pre- versus post-intervention). KEY RESULTS: Across 1527 RAFs, there were significant differences in entrustment ratings between women and men standardized residents (2.29 versus 2.54, respectively, p < 0.001) after correction for resident skill level. As compared to men, feminine terms were more common for comments of what the resident did poorly among women residents (ß 0.45, CI 0.12-0.78, p 0.01). This persisted despite adjusting for the faculty's entrustment ratings. There were no other significant linguistic differences by gender. CONCLUSIONS: Contrasting prior studies, we found entrustment rating differences in a simulated WBA which persisted after adjusting for the resident's scripted performance. There were also linguistic differences by gender after adjusting for entrustment ratings, with feminine terms being used more frequently in comments about women in some, but not all narrative comments.

The Association of Subspecialty and Sex with Industry Payments to Internal Medicine Physicians Who Recently Completed Training.

BACKGROUND: Financial relationships with drug and medical device companies may impact quality of care and academic research. However, little is known when and how these financial relationships develop among newly independent physicians who recently completed from residency or fellowship programs in internal medicine (IM). OBJECTIVE: To compare patterns of industry payments among IM graduates. DESIGN: Retrospective, observational cohort study. SUBJECTS: IM graduates from residency or fellowship programs between January 2015 and December 2019. MAIN MEASURES: We analyzed Open Payments reports made between July 2015 and June 2021 to recent graduates of U.S. Accreditation Council for Graduate Medical Education (ACGME)-accredited residency and fellowship programs in IM. The primary outcome was general payments accepted by these physicians, stratified by procedural (i.e., critical care medicine/pulmonary medicine, cardiac/cardiovascular disease, and gastroenterology) and non-procedural (i.e., infectious disease, general internal medicine, and other specialties) subspecialties. The secondary outcomes included general payments stratified by sex and age at residency or fellowship training completion. KEY RESULTS: There were 41,669 IM physicians with a median age of 33.0 years. In the first 3 years after completion, the proportion of physicians accepting any general payments was 72.6%, 91.9%, and 86.8% in Critical Care Medicine/Pulmonary Medicine, Cardiac/Cardiovascular Disease, and Gastroenterology, compared to 56.1%, 52.6%, and 52.3% in Infectious Disease, General Internal Medicine, and Other Specialties (p<0.0001). After adjusting for confounding variables, the procedural group showed an increased hazard ratio (HR) for accepting any general payments and at least $5000 of general payments compared to the non-procedural group. The HRs of accepting any general payments in the procedural subspecialty were 2.26 (95% CI, 2.11-2.42) and 2.83 (95% CI, 2.70-2.97) in female and male physicians, respectively (p-value < 0.0001). CONCLUSION: Industry financial relationships among newly independent physicians in IM exist immediately after completion of training and are influenced by subspecialty, sex, and age.

Programmatic Assessment of Clinical Reasoning: New Opportunities to Meet an Ongoing Challenge.

Issue: Clinical reasoning is essential to physicians' competence, yet assessment of clinical reasoning remains a significant challenge. Clinical reasoning is a complex, evolving, non-linear, context-driven, and content-specific construct which arguably cannot be assessed at one point in time or with a single method. This has posed challenges for educators for many decades, despite significant development of individual assessment methods. Evidence: Programmatic assessment is a systematic assessment approach that is gaining momentum across health professions education. Programmatic assessment, and in particular assessment for learning, is well-suited to address the challenges with clinical reasoning assessment. Several key principles of programmatic assessment are particularly well-aligned with developing a system to assess clinical reasoning: longitudinality, triangulation, use of a mix of assessment methods, proportionality, implementation of intermediate evaluations/reviews with faculty coaches, use of assessment for feedback, and increase in learners' agency. Repeated exposure and measurement are critical to develop a clinical reasoning assessment narrative, thus the assessment approach should optimally be longitudinal, providing multiple opportunities for growth and development. Triangulation provides a lens to assess the multidimensionality and contextuality of clinical reasoning and that of its different, yet related components, using a mix of different assessment methods. Proportionality ensures the richness of information on which to draw conclusions is commensurate with the stakes of the decision. Coaching facilitates the development of a feedback culture and allows to assess growth over time, while enhancing learners' agency. Implications: A programmatic assessment model of clinical reasoning that is developmentally oriented, optimizes learning though feedback and coaching, uses multiple assessment methods, and provides opportunity for meaningful triangulation of data can help address some of the challenges of clinical reasoning assessment.

Competency-based medical education: The spark to ignite healthcare's escape fire.

High-value care is what patients deserve and what healthcare professionals should deliver. However, it is not what happens much of the time. Quality improvement master Dr. Don Berwick argued more than two decades ago that American healthcare needs an escape fire, which is a new way of seeing and acting in a crisis situation. While coined in the U.S. context, the analogy applies in other Western healthcare contexts as well. Therefore, in this paper, the authors revisit Berwick's analogy, arguing that medical education can, and should, provide the spark for such an escape fire across the globe. They assert that medical education can achieve this by fully embracing competency-based medical education (CBME) as a way to place medicine's focus on the patient. CBME targets training outcomes that prepare graduates to optimize patient care. The authors use the escape fire analogy to argue that medical educators must drop long-held approaches and tools; treat CBME implementation as an adaptive challenge rather than a technical fix; demand genuine, rich discussions and engagement about the path forward; and, above all, center the patient in all they do.

Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship.

Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.

The Use of Learning Analytics to Enable Detection of Underperforming Trainees: An Analysis of National Vascular Surgery Trainee ACGME Milestones Assessment Data.

OBJECTIVE: This study aims to investigate at-risk scores of semiannual Accreditation Council for Graduate Medical Education (ACGME) Milestone ratings for vascular surgical trainees' final achievement of competency targets. SUMMARY BACKGROUND DATA: ACGME Milestones assessments have been collected since 2015 for Vascular Surgery. It is unclear whether milestone ratings throughout training predict achievement of recommended performance targets upon graduation. METHODS: National ACGME Milestones data were utilized for analyses. All trainees completing 2-year vascular surgery fellowships in June 2018 and 5-year integrated vascular surgery residencies in June 2019 were included. A generalized estimating equations model was used to obtain at-risk scores for each of the 31 subcompetencies by semiannual review periods, to estimate the probability of trainees achieving the recommended graduation target based on their previous ratings. RESULTS: A total of 122 vascular surgery fellows (VSFs) (95.3%) and 52 integrated vascular surgery residents (IVSRs) (100%) were included. VSFs and IVSRs did not achieve level 4.0 competency targets at a rate of 1.6% to 25.4% across subcompetencies, which was not significantly different between the 2 groups for any of the subcompetencies ( P = 0.161-0.999). Trainees were found to be at greater risk of not achieving competency targets when lower milestone ratings were assigned, and at later time-points in training. At a milestone rating of 2.5, with 1 year remaining before graduation, the at-risk score for not achieving the target level 4.0 milestone ranged from 2.9% to 77.9% for VSFs and 33.3% to 75.0% for IVSRs. CONCLUSION: The ACGME Milestones provide early diagnostic and predictive information for vascular surgery trainees' achievement of competence at completion of training.

A consensus statement on the meaning, value and utility of training programme outcomes, with specific reference to anaesthesiology: A consensus statement on training programme outcomes.

It is timely and necessary to consider what Postgraduate Medical Training Programme outcomes are, how they are defined and revised over time, and how they can be used to align health professional performance with the healthcare needs of society. This article which addresses those issues, with specific reference to training in anaesthesiology, was prepared using a modified nominal group (or expert panel) approach.

Expert consensus on the conceptual alignment of Accreditation Council for Graduate Medical Education competencies with patient outcomes after common vascular surgical procedures.

BACKGROUND: The quality and effectiveness of vascular surgery education should be evaluated based on patient care outcomes. To investigate predictive associations between trainee performance and subsequent patient outcomes, a critical first step is to determine the conceptual alignment of educational competencies with clinical outcomes in practice. We sought to generate expert consensus on the conceptual alignment of the Accreditation Council for Graduate Medical Education (ACGME) Vascular Surgery subcompetencies with patient care outcomes across different Vascular Quality Initiative (VQI) registries. METHODS: A national panel of vascular surgeons with expertise in both clinical care and education were recruited to participate in a modified Delphi expert consensus building process to map ACGME Vascular Surgery subcompetencies (educational markers of resident performance) to VQI clinical modules (patient outcomes). A master list of items for rating was created, including the 31 ACGME Vascular Surgery subcompetencies and 8 VQI clinical registries (endovascular abdominal aortic aneurysm repair, open abdominal aortic aneurysm, thoracic endovascular aortic repair, carotid endarterectomy, carotid artery stent, infrainguinal, suprainguinal, and peripheral vascular intervention). These items were entered into an iterative Delphi process. Positive consensus was reached when 75% or more of the participants ranked an item as mandatory. Intraclass correlations (ICCs) were used to evaluate consistency between experts for each Delphi round. RESULTS: A total of 13 experts who contributed to the development of the Vascular Surgery Milestones participated; 12 experts (92%) participated in both rounds of the Delphi process. Two rounds of Delphi were conducted, as suggested by excellent expert agreement (round 1, ICC = 0.79 [95% confidence interval, 0.74-0.84]; round 2, ICC = 0.97 [95% confidence interval, 0.960-.98]). Using the predetermined consensus cutoff threshold, the Delphi process reduced the number of subcompetencies mapped to patient care outcomes from 31 to a range of 9 to 15 across the 8 VQI clinical registries. Practice-based learning and improvement, and professionalism subcompetencies were identified as less relevant to patient outcome variables captured by the VQI registries after the final round, and the only the systems-based practice subcompetency that was identified as relevant was radiation safety in two of the endovascular registries. CONCLUSIONS: A national panel of vascular surgeon experts reported a high degree of agreement on the relevance of ACGME subcompetencies to patient care outcomes as captured in the VQI clinical registry. Systems-based practice, practice-based learning and improvement, and professionalism competencies were identified as less relevant to patient outcomes after specific surgical procedures.

A multispecialty ethnographic study of clinical competency committees (CCCs).

PURPOSE: Clinical competency committees (CCCs) assess residents' performance on their specialty specific milestones, however there is no 'one-size fits all' blueprint for accomplishing this. Thus, CCCs have had to develop their own procedures. The goal of this study was to examine these efforts to assist new programs embarking on this venture and established programs looking to improve their CCC practices and processes. METHODS: We purposefully sampled CCCs across multiple specialties and institutions. Data from three sources were triangulated: (1) online demographic survey, (2) ethnographic observations of CCC meetings and (3) post-observation semi-structured interviews with the program director and/or CCC chairperson. Template analysis was used to build the coding structure. RESULTS: Sixteen observations were completed with 15 different CCCs at 9 institutions. Three main thematic categories that impact the operations of CCCs emerged: (1) Membership structure and members roles, (2) Roles of the CCC in residency and 3) CCC processes, including trainee presentation to the committee and decision-making. While effective practices were observed, substantial variation existed in all three thematic areas. CONCLUSIONS: While CCCs used some known effective practices, substantial variation in structure and processes was notable across CCCs. Future work should explore the impact of this variation on educational outcomes.

Competency-based medical education across the continuum: How well aligned are medical school EPAs to residency milestones?

INTRODUCTION: Competency-based medical education (CBME) provides a framework for describing learner progression throughout training. However, specific approaches to CBME implementation vary widely across educational settings. Alignment between various methods used across the continuum is critical to support transitions and assess learner performance. The purpose of this study was to investigate alignment between CBME frameworks used in undergraduate medical education (UME) and graduate medical education (GME) settings using the US context as a model. METHOD: The authors analyzed content from the core entrustable professional activities for entering residency (Core EPAs; UME model) and residency milestones (GME model). From that analysis, they performed a series of cross-walk activities to investigate alignment between frameworks. After independent review, authors discussed findings until consensus was reached. RESULTS: Some alignment was found for activities associated with history taking, physical examination, differential diagnosis, patient safety, and interprofessional care; however, there were far more examples of misalignment. CONCLUSIONS: These findings highlight challenges creating alignment of assessment frameworks across the continuum of training. The importance of these findings includes implications for assessment and persistence of the educational gap across UME and GME. The authors provide four next steps to improve upon the continuum of education.

Doctor@Home: Through a Telemedicine Co-production and Co-learning Journey.

Telemedicine and remote visits are becoming more and more popular in several medical disciplines, including oncology. The Covid-19 pandemic has enhanced the need to continue to meet patients' ambulatory care necessities ensuring social distancing and limiting the access to clinical facilities. The National Cancer Institute of Aviano, Italy, has recently launched a program called "Doctor @ Home" (D@H). The pillars of the program are the co-production of the oncological care and the co-learning approach, which sees the clinical staff "hand in hand" with patients to maximize the outcome of the care, trying to take advantage of the new tools offered by modern technologies.

Correlations between Accreditation Council for Graduate Medical Education Obstetrics and Gynecology Milestones and American Board of Obstetrics and Gynecology qualifying examination scores: an initial validity study.

BACKGROUND: Since the launch of the Outcome Project in 2001, the graduate medical education community has been working to implement the 6 general competencies. In 2014, all Obstetrics and Gynecology residency programs implemented specialty-specific milestones to advance competency-based assessment. Each clinical competency committee of the Obstetrics and Gynecology program assesses all residents twice a year on the milestones. These data are reported to the Accreditation Council for Graduate Medical Education as part of a continuous quality improvement effort in graduate medical education. OBJECTIVE: This study aimed to evaluate the correlation between the Accreditation Council for Graduate Medical Education Obstetrics and Gynecology Milestones and residency program graduates' performance on the American Board of Obstetrics and Gynecology qualifying (written) examination. STUDY DESIGN: We conducted a validity study of all graduating (postgraduate year 4) Obstetrics and Gynecology residents in 2017 within Accreditation Council for Graduate Medical Education-accredited United States training programs (1260 residents from 242 programs). This cohort of residents began receiving milestone assessments during their postgraduate year 2 in 2014; the first-year milestones were implemented for all Accreditation Council for Graduate Medical Education-accredited Obstetrics and Gynecology programs. This cohort completed their sixth and final milestone assessment at graduation in June 2017 for a total of 6 periods of milestone assessments. Data regarding each resident's milestone ratings in each of the 28 Accreditation Council for Graduate Medical Education subcompetencies for Obstetrics and Gynecology were assessed for their association with candidates' American Board of Obstetrics and Gynecology qualifying examination scores using a generalized estimating equation regression model. RESULTS: Data were available and analyzed from 1184 residents from 240 programs, representing 94% of the total academic year 2017 graduates of Obstetrics and Gynecology residency training programs. There was a substantial association between most milestone ratings at the 6 assessment points and candidates' performance on the American Board of Obstetrics and Gynecology qualifying examination. The strongest associations with the American Board of Obstetrics and Gynecology were within all 7 of the subcompetencies of Medical Knowledge (range of slope correlation coefficients at final milestone ratings 3.84-5.17; slope coefficients can be interpreted as the gain in qualifying examination points per unit increase in milestone level). At the final milestone assessment, but more modest associations with the American Board of Obstetrics and Gynecology qualifying examination scores were also seen with 9 of the 11 Patient Care and Procedural Skills subcompetencies, the 2 of 2 Practice-Based Learning and Improvement subcompetencies, the 2 of 2 Systems-Based Practice subcompetencies, and 2 of the 3 Professionalism subcompetencies. Only 1 of the 3 Interpersonal and Communication Skills subcompetencies was associated with American Board of Obstetrics and Gynecology qualifying examination scores. CONCLUSION: The pattern of associations between the qualifying examination scores and milestone ratings for the 2017 graduating cohort of Obstetrics and Gynecology residents followed a logical pattern, with the strongest associations seen in Medical Knowledge, and lower to no associations in subcompetencies not as effectively assessed on multiple-choice examinations. Although some positive associations were noted for non-Medical Knowledge milestones, these associations could be caused by correlational rating errors with further study needed to better understand these patterns.

Using Learning Analytics to Examine Achievement of Graduation Targets for Systems-Based Practice and Practice-Based Learning and Improvement: A National Cohort of Vascular Surgery Fellows.

BACKGROUND: Surgeons provide patient care in complex health care systems and must be able to participate in improving both personal performance and the performance of the system. The Accreditation Council for Graduate Medical Education (ACGME) Vascular Surgery Milestones are utilized to assess vascular surgery fellows' (VSF) achievement of graduation targets in the competencies of Systems Based Practice (SBP) and Practice Based Learning and Improvement (PBLI). We investigate the predictive value of semiannual milestones ratings for final achievement within these competencies at the time of graduation. METHODS: National ACGME milestones data were utilized for analysis. All trainees entering the 2-year vascular surgery fellowship programs in July 2016 were included in the analysis (n = 122). Predictive probability values (PPVs) were obtained for each SBP and PBLI sub-competencies by biannual review periods, to estimate the probability of VSFs not reaching the recommended graduation target based on their previous milestones ratings. RESULTS: The rate of nonachievement of the graduation target level 4.0 on the SBP and PBLI sub-competencies at the time of graduation for VSFs was 13.1-25.4%. At the first time point of assessment, 6 months into the fellowship program, the PPV of the SBP and PBLI milestones for nonachievement of level 4.0 upon graduation ranged from 16.3-60.2%. Six months prior to graduation, the PPVs across the 6 sub-competencies ranged from 14.6-82.9%. CONCLUSIONS: A significant percentage of VSFs do not achieve the ACGME Vascular Surgery Milestone targets for graduation in the competencies of SBP and PBLI, suggesting a need to improve curricula and assessment strategies in these domains across vascular surgery fellowship programs. Reported milestones levels across all time point are predictive of ultimate achievement upon graduation and should be utilized to provide targeted feedback and individualized learning plans to ensure graduates are prepared to engage in personal and health care system improvement once in unsupervised practice.

Why Residencies Should Fly: Towards a Logical Approach to Duty Hour Reform.

Since 2011, aviation has revolutionized their approach to safety. The aviation industry has adopted a multi-faceted approach to improve safety through decreasing duty hour limits and implementing processes to mitigate fatigue-related errors as well as creating cultural shifts in responsibility for safety. These changes have been guided by data generated by quality-improvement methodology. In contrast, duty hour limits in graduate medical education have not yet seen dramatic data-driven reform. Key advancements in aviation fatigue mitigation and implications for residency education are explored in this article. Scientifically based processes to optimize duty hours, quality-improvement strategies to iteratively monitor and reform duty limits, systematic change focusing on a just culture, and financial disincentives and incentives as a catalyst for change are discussed.

Entrustment decisions: Implications for curriculum development and assessment.

With increased interest in the use of entrustable professional activities (EPAs) in undergraduate and postgraduate medical education, comes questions about their implications for curriculum development and assessment. This paper addresses some of those questions, discussed at a symposium of the 2017 conference of AMEE, by presenting the components of an EPA, describing their importance and application, identifying their implications for assessment, and pinpointing some of challenges they pose in undergraduate and postgraduate settings. It defines entrustment, describes the three levels of trust, and presents trainee and supervisor factors that influence it as well as perceived benefits, and risks. Two aspects of EPAs have implications for assessment: units of professional practice and decisions based on entrustment, which impact an assessment's blueprint, test methods, scores, and standards. In an undergraduate setting EPAs have great appeal, but work is needed to identify and develop a robust assessment system for core EPAs. At the postgraduate level, there is tension between the granularity of the competencies and the integrated nature of the EPAs. Even though work remains, EPAs offer an important step in the evolution of competency-based education.

Mapping clinical reasoning literature across the health professions: a scoping review.

BACKGROUND: Clinical reasoning is at the core of health professionals' practice. A mapping of what constitutes clinical reasoning could support the teaching, development, and assessment of clinical reasoning across the health professions. METHODS: We conducted a scoping study to map the literature on clinical reasoning across health professions literature in the context of a larger Best Evidence Medical Education (BEME) review on clinical reasoning assessment. Seven databases were searched using subheadings and terms relating to clinical reasoning, assessment, and Health Professions. Data analysis focused on a comprehensive analysis of bibliometric characteristics and the use of varied terminology to refer to clinical reasoning. RESULTS: Literature identified: 625 papers spanning 47 years (1968-2014), in 155 journals, from 544 first authors, across eighteen Health Professions. Thirty-seven percent of papers used the term clinical reasoning; and 110 other terms referring to the concept of clinical reasoning were identified. Consensus on the categorization of terms was reached for 65 terms across six different categories: reasoning skills, reasoning performance, reasoning process, outcome of reasoning, context of reasoning, and purpose/goal of reasoning. Categories of terminology used differed across Health Professions and publication types. DISCUSSION: Many diverse terms were present and were used differently across literature contexts. These terms likely reflect different operationalisations, or conceptualizations, of clinical reasoning as well as the complex, multi-dimensional nature of this concept. We advise authors to make the intended meaning of 'clinical reasoning' and associated terms in their work explicit in order to facilitate teaching, assessment, and research communication.

Managing tensions in assessment: moving beyond either-or thinking.

CONTEXT: In health professions education, assessment systems are bound to be rife with tensions as they must fulfil formative and summative assessment purposes, be efficient and effective, and meet the needs of learners and education institutes, as well as those of patients and health care organisations. The way we respond to these tensions determines the fate of assessment practices and reform. In this study, we argue that traditional 'fix-the-problem' approaches (i.e. either-or solutions) are generally inadequate and that we need alternative strategies to help us further understand, accept and actually engage with the multiple recurring tensions in assessment programmes. METHODS: Drawing from research in organisation science and health care, we outline how the Polarity Thinking™ model and its 'both-and' approach offer ways to systematically leverage assessment tensions as opportunities to drive improvement, rather than as intractable problems. In reviewing the assessment literature, we highlight and discuss exemplars of specific assessment polarities and tensions in educational settings. Using key concepts and principles of the Polarity Thinking™ model, and two examples of common tensions in assessment design, we describe how the model can be applied in a stepwise approach to the management of key polarities in assessment. DISCUSSION: Assessment polarities and tensions are likely to surface with the continued rise of complexity and change in education and health care organisations. With increasing pressures of accountability in times of stretched resources, assessment tensions and dilemmas will become more pronounced. We propose to add to our repertoire of strategies for managing key dilemmas in education and assessment design through the adoption of the polarity framework. Its 'both-and' approach may advance our efforts to transform assessment systems to meet complex 21st century education, health and health care needs.