The Predictive Performance of General Surgery Milestones on Post-Graduation Outcomes.

OBJECTIVE: To establish whether Accreditation Council for Graduate Medical Education Milestones predict future performance of general surgery trainees. SUMMARY BACKGROUND DATA: Milestones provide bi-annual assessments of trainee progress across six competencies. It is unknown whether the Milestones predict surgeon performance after the transition to independent practice. METHODS: We performed a retrospective cohort study of surgeons with complete Milestone assessments in the fourth and fifth clinical years who treated patients in acute care hospitals within Florida, New York, and Pennsylvania, 2015-2018. To account for the multiple ways in which the Milestone assessments might predict post-graduation outcomes, we included 120 Milestones features in our elastic net machine learning models. The primary outcome was risk-adjusted patient death or serious morbidity. RESULTS: 278 general surgeons were included in the study. Milestone assessments 6-months into the fourth clinical year displayed a normal score distribution while multicollinearity and low score discrimination at the final assessment period were detected. Individual Milestones features from the Patient Care, Professionalism, and Systems-based Practice domains were most predictive of patient-related outcomes. For example, surgeons with worse patient outcomes had significantly lower scores in Patient Care 3 when compared to surgeons with better patient outcomes (High DSM, yes: 2.86 vs. no: 3.04, P=0.011). CONCLUSIONS: The Milestones features that were most predictive of better patient outcomes related to intraoperative skills, ethical principles, and patient navigation and safety, measured 12-18 months prior to graduation. The development of a parsimonious set of evidence-based Milestones that better correlate with surgeon experience could enhance surgical education.

Gender Differences in Work-Based Assessment Scores and Narrative Comments After Direct Observation.

BACKGROUND: While some prior studies of work-based assessment (WBA) numeric ratings have not shown gender differences, they have been unable to account for the true performance of the resident or explore narrative differences by gender. OBJECTIVE: To explore gender differences in WBA ratings as well as narrative comments (when scripted performance was known). DESIGN: Secondary analysis of WBAs obtained from a randomized controlled trial of a longitudinal rater training intervention in 2018-2019. Participating faculty (n = 77) observed standardized resident-patient encounters and subsequently completed rater assessment forms (RAFs). SUBJECTS: Participating faculty in longitudinal rater training. MAIN MEASURES: Gender differences in mean entrustment ratings (4-point scale) were assessed with multivariable regression (adjusted for scripted performance, rater and resident demographics, and the interaction between study arm and time period [pre- versus post-intervention]). Using pre-specified natural language processing categories (masculine, feminine, agentic, and communal words), multivariable linear regression was used to determine associations of word use in the narrative comments with resident gender, race, and skill level, faculty demographics, and interaction between the study arm and the time period (pre- versus post-intervention). KEY RESULTS: Across 1527 RAFs, there were significant differences in entrustment ratings between women and men standardized residents (2.29 versus 2.54, respectively, p < 0.001) after correction for resident skill level. As compared to men, feminine terms were more common for comments of what the resident did poorly among women residents (ß 0.45, CI 0.12-0.78, p 0.01). This persisted despite adjusting for the faculty's entrustment ratings. There were no other significant linguistic differences by gender. CONCLUSIONS: Contrasting prior studies, we found entrustment rating differences in a simulated WBA which persisted after adjusting for the resident's scripted performance. There were also linguistic differences by gender after adjusting for entrustment ratings, with feminine terms being used more frequently in comments about women in some, but not all narrative comments.

Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship.

Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.

A multispecialty ethnographic study of clinical competency committees (CCCs).

PURPOSE: Clinical competency committees (CCCs) assess residents' performance on their specialty specific milestones, however there is no 'one-size fits all' blueprint for accomplishing this. Thus, CCCs have had to develop their own procedures. The goal of this study was to examine these efforts to assist new programs embarking on this venture and established programs looking to improve their CCC practices and processes. METHODS: We purposefully sampled CCCs across multiple specialties and institutions. Data from three sources were triangulated: (1) online demographic survey, (2) ethnographic observations of CCC meetings and (3) post-observation semi-structured interviews with the program director and/or CCC chairperson. Template analysis was used to build the coding structure. RESULTS: Sixteen observations were completed with 15 different CCCs at 9 institutions. Three main thematic categories that impact the operations of CCCs emerged: (1) Membership structure and members roles, (2) Roles of the CCC in residency and 3) CCC processes, including trainee presentation to the committee and decision-making. While effective practices were observed, substantial variation existed in all three thematic areas. CONCLUSIONS: While CCCs used some known effective practices, substantial variation in structure and processes was notable across CCCs. Future work should explore the impact of this variation on educational outcomes.

Competency-based medical education across the continuum: How well aligned are medical school EPAs to residency milestones?

INTRODUCTION: Competency-based medical education (CBME) provides a framework for describing learner progression throughout training. However, specific approaches to CBME implementation vary widely across educational settings. Alignment between various methods used across the continuum is critical to support transitions and assess learner performance. The purpose of this study was to investigate alignment between CBME frameworks used in undergraduate medical education (UME) and graduate medical education (GME) settings using the US context as a model. METHOD: The authors analyzed content from the core entrustable professional activities for entering residency (Core EPAs; UME model) and residency milestones (GME model). From that analysis, they performed a series of cross-walk activities to investigate alignment between frameworks. After independent review, authors discussed findings until consensus was reached. RESULTS: Some alignment was found for activities associated with history taking, physical examination, differential diagnosis, patient safety, and interprofessional care; however, there were far more examples of misalignment. CONCLUSIONS: These findings highlight challenges creating alignment of assessment frameworks across the continuum of training. The importance of these findings includes implications for assessment and persistence of the educational gap across UME and GME. The authors provide four next steps to improve upon the continuum of education.

Doctor@Home: Through a Telemedicine Co-production and Co-learning Journey.

Telemedicine and remote visits are becoming more and more popular in several medical disciplines, including oncology. The Covid-19 pandemic has enhanced the need to continue to meet patients' ambulatory care necessities ensuring social distancing and limiting the access to clinical facilities. The National Cancer Institute of Aviano, Italy, has recently launched a program called "Doctor @ Home" (D@H). The pillars of the program are the co-production of the oncological care and the co-learning approach, which sees the clinical staff "hand in hand" with patients to maximize the outcome of the care, trying to take advantage of the new tools offered by modern technologies.

Correlations between Accreditation Council for Graduate Medical Education Obstetrics and Gynecology Milestones and American Board of Obstetrics and Gynecology qualifying examination scores: an initial validity study.

BACKGROUND: Since the launch of the Outcome Project in 2001, the graduate medical education community has been working to implement the 6 general competencies. In 2014, all Obstetrics and Gynecology residency programs implemented specialty-specific milestones to advance competency-based assessment. Each clinical competency committee of the Obstetrics and Gynecology program assesses all residents twice a year on the milestones. These data are reported to the Accreditation Council for Graduate Medical Education as part of a continuous quality improvement effort in graduate medical education. OBJECTIVE: This study aimed to evaluate the correlation between the Accreditation Council for Graduate Medical Education Obstetrics and Gynecology Milestones and residency program graduates' performance on the American Board of Obstetrics and Gynecology qualifying (written) examination. STUDY DESIGN: We conducted a validity study of all graduating (postgraduate year 4) Obstetrics and Gynecology residents in 2017 within Accreditation Council for Graduate Medical Education-accredited United States training programs (1260 residents from 242 programs). This cohort of residents began receiving milestone assessments during their postgraduate year 2 in 2014; the first-year milestones were implemented for all Accreditation Council for Graduate Medical Education-accredited Obstetrics and Gynecology programs. This cohort completed their sixth and final milestone assessment at graduation in June 2017 for a total of 6 periods of milestone assessments. Data regarding each resident's milestone ratings in each of the 28 Accreditation Council for Graduate Medical Education subcompetencies for Obstetrics and Gynecology were assessed for their association with candidates' American Board of Obstetrics and Gynecology qualifying examination scores using a generalized estimating equation regression model. RESULTS: Data were available and analyzed from 1184 residents from 240 programs, representing 94% of the total academic year 2017 graduates of Obstetrics and Gynecology residency training programs. There was a substantial association between most milestone ratings at the 6 assessment points and candidates' performance on the American Board of Obstetrics and Gynecology qualifying examination. The strongest associations with the American Board of Obstetrics and Gynecology were within all 7 of the subcompetencies of Medical Knowledge (range of slope correlation coefficients at final milestone ratings 3.84-5.17; slope coefficients can be interpreted as the gain in qualifying examination points per unit increase in milestone level). At the final milestone assessment, but more modest associations with the American Board of Obstetrics and Gynecology qualifying examination scores were also seen with 9 of the 11 Patient Care and Procedural Skills subcompetencies, the 2 of 2 Practice-Based Learning and Improvement subcompetencies, the 2 of 2 Systems-Based Practice subcompetencies, and 2 of the 3 Professionalism subcompetencies. Only 1 of the 3 Interpersonal and Communication Skills subcompetencies was associated with American Board of Obstetrics and Gynecology qualifying examination scores. CONCLUSION: The pattern of associations between the qualifying examination scores and milestone ratings for the 2017 graduating cohort of Obstetrics and Gynecology residents followed a logical pattern, with the strongest associations seen in Medical Knowledge, and lower to no associations in subcompetencies not as effectively assessed on multiple-choice examinations. Although some positive associations were noted for non-Medical Knowledge milestones, these associations could be caused by correlational rating errors with further study needed to better understand these patterns.

Managing tensions in assessment: moving beyond either-or thinking.

CONTEXT: In health professions education, assessment systems are bound to be rife with tensions as they must fulfil formative and summative assessment purposes, be efficient and effective, and meet the needs of learners and education institutes, as well as those of patients and health care organisations. The way we respond to these tensions determines the fate of assessment practices and reform. In this study, we argue that traditional 'fix-the-problem' approaches (i.e. either-or solutions) are generally inadequate and that we need alternative strategies to help us further understand, accept and actually engage with the multiple recurring tensions in assessment programmes. METHODS: Drawing from research in organisation science and health care, we outline how the Polarity Thinking™ model and its 'both-and' approach offer ways to systematically leverage assessment tensions as opportunities to drive improvement, rather than as intractable problems. In reviewing the assessment literature, we highlight and discuss exemplars of specific assessment polarities and tensions in educational settings. Using key concepts and principles of the Polarity Thinking™ model, and two examples of common tensions in assessment design, we describe how the model can be applied in a stepwise approach to the management of key polarities in assessment. DISCUSSION: Assessment polarities and tensions are likely to surface with the continued rise of complexity and change in education and health care organisations. With increasing pressures of accountability in times of stretched resources, assessment tensions and dilemmas will become more pronounced. We propose to add to our repertoire of strategies for managing key dilemmas in education and assessment design through the adoption of the polarity framework. Its 'both-and' approach may advance our efforts to transform assessment systems to meet complex 21st century education, health and health care needs.

Trust in Group Decisions: a scoping review.

BACKGROUND: Trust is a critical component of competency committees given their high-stakes decisions. Research from outside of medicine on group trust has not focused on trust in group decisions, and "group trust" has not been clearly defined. The purpose was twofold: to examine the definition of trust in the context of group decisions and to explore what factors may influence trust from the perspective of those who rely on competency committees through a proposed group trust model. METHODS: The authors conducted a literature search of four online databases, seeking articles published on trust in group settings. Reviewers extracted, coded, and analyzed key data including definitions of trust and factors pertaining to group trust. RESULTS: The authors selected 42 articles for full text review. Although reviewers found multiple general definitions of trust, they were unable to find a clear definition of group trust and propose the following: a group-directed willingness to accept vulnerability to actions of the members based on the expectation that members will perform a particular action important to the group, encompassing social exchange, collective perceptions, and interpersonal trust. Additionally, the authors propose a model encompassing individual level factors (trustor and trustee), interpersonal interactions, group level factors (structure and processes), and environmental factors. CONCLUSIONS: Higher degrees of trust at the individual and group levels have been associated with attitudinal and performance outcomes, such as quality of group decisions. Developing a deeper understanding of trust in competency committees may help these committees implement more effective and meaningful processes to make collective decisions.

Physician Experiences With High Value Care in Internal Medicine Residency: Mixed-Methods Study of 2003-2013 Residency Graduates.

Phenomenon: High healthcare costs and relatively poor health outcomes in the United States have led to calls to improve the teaching of high value care (defined as care that balances potential benefits of interventions with their harms including costs) to physicians-in-training. Numerous interventions to increase high value care in graduate medical education were implemented at the national and local levels over the past decade. However, there has been little evaluation of their impact on physician experiences during training and perceived preparedness for practice. We aimed to assess trends in U.S. physician experiences with high value care during residency over the past decade. APPROACH: This mixed-methods study used a cross-sectional survey mailed July 2014 to January 2015 to 902 internists who completed residency in 2003-2013, randomly selected from the American Medical Association Masterfile. Quantitative analyses of survey responses and content analysis of free-text comments submitted by respondents were performed. FINDINGS: A total of 456 physicians (50.6%) responded. Fewer than one fourth reported being exposed to teaching about high value care at least frequently (23.6%, 106/450). Only 43.8% of respondents (193/446) felt prepared to use overtreatment guidelines in conversations with patients, whereas 85.8% (379/447) felt prepared to participate in shared decision making with patients at the conclusion of their training, and 84.4% (380/450) reported practicing generic prescribing. Physicians who completed residency more recently were more likely to report practicing generic prescribing and feeling well prepared to use overtreatment guidelines in conversations with patients (p < .01 for both). Insights: In a national survey, recent U.S. internal medicine residency graduates were more likely to experience high value care during training, which may reflect increased national and local efforts in this area. However, being exposed to high value care as a trainee may not translate into specific tools for practice. In fact, many U.S. internists reported inadequate exposure to prepare them for patient discussions about costs and the use of overtreatment guidelines in practice.

The Internal Medicine Reporting Milestones: Cross-sectional Description of Initial Implementation in U.S. Residency Programs.

BACKGROUND: High-quality assessment of resident performance is needed to guide individual residents' development and ensure their preparedness to provide patient care. To facilitate this aim, reporting milestones are now required across all internal medicine (IM) residency programs. OBJECTIVE: To describe initial milestone ratings for the population of IM residents by IM residency programs. DESIGN: Cross-sectional study. SETTING: IM residency programs. PARTICIPANTS: All IM residents whose residency program directors submitted milestone data at the end of the 2013-2014 academic year. MEASUREMENTS: Ratings addressed 6 competencies and 22 subcompetencies. A rating of "not assessable" indicated insufficient information to evaluate the given subcompetency. Descriptive statistics were calculated to describe ratings across competencies and training years. RESULTS: Data were available for all 21 774 U.S. IM residents from all 383 programs. Overall, 2889 residents (1621 in postgraduate year 1 [PGY-1], 902 in PGY-2, and 366 in PGY-3) had at least 1 subcompetency rated as not assessable. Summaries of average ratings by competency and training year showed higher ratings for PGY-3 residents in all competencies. Overall ratings for each of the 6 individual competencies showed that fewer than 1% of third-year residents were rated as "unsatisfactory" or "conditional on improvement." However, when subcompetency milestone ratings were used, 861 residents (12.8%) who successfully completed training had at least 1 competency with all corresponding subcompetencies graded below the threshold of "readiness for unsupervised practice." LIMITATION: Data were derived from a point in time in the first reporting period in which milestones were used. CONCLUSION: The initial milestone-based evaluations of IM residents nationally suggest that documenting developmental progression of competency is possible over training years. Subcompetencies may identify areas in which residents might benefit from additional feedback and experience. Future work is needed to explore how milestones are used to support residents' development and enhance residency curricula. PRIMARY FUNDING SOURCE: None.

Implementing competency-based medical education: Moving forward.

For more than 60 years, competency-based education has been proposed as an approach to education in many disciplines. In medical education, interest in CBME has grown dramatically in the last decade. This editorial introduces a series of papers that resulted from summits held in 2013 and 2016 by the International CBME Collaborators, a scholarly network whose members are interested in developing competency-based approaches to preparing the next generation of health professionals. An overview of the papers is given, as well as a summary of landmarks in the conceptual evolution and implementation of CBME. This series follows on a first collection of papers published by the International CBME Collaborators in Medical Teacher in 2010.

A call to action: The controversy of and rationale for competency-based medical education.

Although medical education has enjoyed many successes over the last century, there is a recognition that health care is too often unsafe and of poor quality. Errors in diagnosis and treatment, communication breakdowns, poor care coordination, inappropriate use of tests and procedures, and dysfunctional collaboration harm patients and families around the world. These issues reflect on our current model of medical education and raise the question: Are physicians being adequately prepared for twenty-first century practice? Multiple reports have concluded the answer is "no." Concurrent with this concern is an increasing interest in competency-based medical education (CBME) as an approach to help reform medical education. The principles of CBME are grounded in providing better and safer care. As interest in CBME has increased, so have criticisms of the movement. This article summarizes and addresses objections and challenges related to CBME. These can provide valuable feedback to improve CBME implementation and avoid pitfalls. We strongly believe medical education reform should not be reduced to an "either/or" approach, but should blend theories and approaches to suit the needs and resources of the populations served. The incorporation of milestones and entrustable professional activities within existing competency frameworks speaks to the dynamic evolution of CBME, which should not be viewed as a fixed doctrine, but rather as a set of evolving concepts, principles, tools, and approaches that can enable important reforms in medical education that, in turn, enable the best outcomes for patients.

Core principles of assessment in competency-based medical education.

The meaningful assessment of competence is critical for the implementation of effective competency-based medical education (CBME). Timely ongoing assessments are needed along with comprehensive periodic reviews to ensure that trainees continue to progress. New approaches are needed to optimize the use of multiple assessors and assessments; to synthesize the data collected from multiple assessors and multiple types of assessments; to develop faculty competence in assessment; and to ensure that relationships between the givers and receivers of feedback are appropriate. This paper describes the core principles of assessment for learning and assessment of learning. It addresses several ways to ensure the effectiveness of assessment programs, including using the right combination of assessment methods and conducting careful assessor selection and training. It provides a reconceptualization of the role of psychometrics and articulates the importance of a group process in determining trainees' progress. In addition, it notes that, to reach its potential as a driver in trainee development, quality care, and patient safety, CBME requires effective information management and documentation as well as ongoing consideration of ways to improve the assessment system.

Association of Physician Certification in Interventional Cardiology With In-Hospital Outcomes of Percutaneous Coronary Intervention.

BACKGROUND: The value of American Board of Internal Medicine certification has been questioned. We evaluated the Association of Interventional Cardiology certification with in-hospital outcomes of patients undergoing percutaneous coronary intervention (PCI) in 2010. METHODS AND RESULTS: We identified physicians who performed ≥10 PCIs in 2010 in the CathPCI Registry and determined interventional cardiology (ICARD) certification status using American Board of Internal Medicine data. We compared in-hospital outcomes of patients treated by certified and noncertified physicians using hierarchical multivariable models adjusted for differences in patient characteristics and PCI volume. Primary end points were all-cause in-hospital mortality and bleeding complications. Secondary end points included emergency coronary artery bypass grafting, vascular complications, and a composite of any adverse outcome. With 510,708 PCI procedures performed by 5175 physicians, case mix and unadjusted outcomes were similar among certified and noncertified physicians. The adjusted risks of in-hospital mortality (odds ratio, 1.10; 95% confidence interval, 1.02-1.19) and emergency coronary artery bypass grafting (odds ratio, 1.32; 95% confidence interval, 1.12-1.56) were higher in the non-ICARD-certified group, but the risks of bleeding and vascular complications and the composite end point were not statistically significantly different between groups. CONCLUSIONS: We did not observe a consistent association between ICARD certification and the outcomes of PCI procedures. Although there was a significantly higher risk of mortality and emergency coronary artery bypass grafting in patients treated by non-ICARD-certified physicians, the risks of vascular complications and bleeding were similar. Our findings suggest that ICARD certification status alone is not a strong predictor of patient outcomes and indicate a need to enhance the value of subspecialty certification.

Functional neuroimaging correlates of thinking flexibility and knowledge structure in memory: Exploring the relationships between clinical reasoning and diagnostic thinking.

BACKGROUND: Diagnostic reasoning involves the thinking steps up to and including arrival at a diagnosis. Dual process theory posits that a physician's thinking is based on both non-analytic or fast, subconscious thinking and analytic thinking that is slower, more conscious, effortful and characterized by comparing and contrasting alternatives. Expertise in clinical reasoning may relate to the two dimensions measured by the diagnostic thinking inventory (DTI): memory structure and flexibility in thinking. AIM: Explored the functional magnetic resonance imaging (fMRI) correlates of these two aspects of the DTI: memory structure and flexibility of thinking. METHODS: Participants answered and reflected upon multiple-choice questions (MCQs) during fMRI. A DTI was completed shortly after the scan. The brain processes associated with the two dimensions of the DTI were correlated with fMRI phases - assessing flexibility in thinking during analytical clinical reasoning, memory structure during non-analytical clinical reasoning and the total DTI during both non-analytical and analytical reasoning in experienced physicians. RESULTS: Each DTI component was associated with distinct functional neuroanatomic activation patterns, particularly in the prefrontal cortex. CONCLUSION: Our findings support diagnostic thinking conceptual models and indicate mechanisms through which cognitive demands may induce functional adaptation within the prefrontal cortex. This provides additional objective validity evidence for the use of the DTI in medical education and practice settings.

Correlations Between Ratings on the Resident Annual Evaluation Summary and the Internal Medicine Milestones and Association With ABIM Certification Examination Scores Among US Internal Medicine Residents, 2013-2014.

Importance: US internal medicine residency programs are now required to rate residents using milestones. Evidence of validity of milestone ratings is needed. Objective: To compare ratings of internal medicine residents using the pre-2015 resident annual evaluation summary (RAES), a nondevelopmental rating scale, with developmental milestone ratings. Design, Setting, and Participants: Cross-sectional study of US internal medicine residency programs in the 2013-2014 academic year, including 21 284 internal medicine residents (7048 postgraduate-year 1 [PGY-1], 7233 PGY-2, and 7003 PGY-3). Exposures: Program director ratings on the RAES and milestone ratings. Main Outcomes and Measures: Correlations of RAES and milestone ratings by training year; correlations of medical knowledge ratings with American Board of Internal Medicine (ABIM) certification examination scores; rating of unprofessional behavior using the 2 systems. Results: Corresponding RAES ratings and milestone ratings showed progressively higher correlations across training years, ranging among competencies from 0.31 (95% CI, 0.29 to 0.33) to 0.35 (95% CI, 0.33 to 0.37) for PGY-1 residents to 0.43 (95% CI, 0.41 to 0.45) to 0.52 (95% CI, 0.50 to 0.54) for PGY-3 residents (all P values <.05). Linear regression showed ratings differed more between PGY-1 and PGY-3 years using milestone ratings than the RAES (all P values <.001). Of the 6260 residents who attempted the certification examination, the 618 who failed had lower ratings using both systems for medical knowledge than did those who passed (RAES difference, -0.9; 95% CI, -1.0 to -0.8; P < .001; milestone medical knowledge 1 difference, -0.3; 95% CI, -0.3 to -0.3; P < .001; and medical knowledge 2 difference, -0.2; 95% CI, -0.3 to -0.2; P < .001). Of the 26 PGY-3 residents with milestone ratings indicating deficiencies on either of the 2 medical knowledge subcompetencies, 12 failed the certification examination. Correlation of RAES ratings for professionalism with residents' lowest professionalism milestone ratings was 0.44 (95% CI, 0.43 to 0.45; P < .001). Conclusions and Relevance: Among US internal medicine residents in the 2013-2014 academic year, milestone-based ratings correlated with RAES ratings but with a greater difference across training years. Both rating systems for medical knowledge correlated with ABIM certification examination scores. Milestone ratings may better detect problems with professionalism. These preliminary findings may inform establishment of the validity of milestone-based assessment.

Implementation of competency-based medical education: are we addressing the concerns and challenges?

CONTEXT: Competency-based medical education (CBME) has emerged as a core strategy to educate and assess the next generation of physicians. Advantages of CBME include: a focus on outcomes and learner achievement; requirements for multifaceted assessment that embraces formative and summative approaches; support of a flexible, time-independent trajectory through the curriculum; and increased accountability to stakeholders with a shared set of expectations and a common language for education, assessment and regulation. OBJECTIVES: Despite the advantages of CBME, numerous concerns and challenges to the implementation of CBME frameworks have been described, including: increased administrative requirements; the need for faculty development; the lack of models for flexible curricula, and inconsistencies in terms and definitions. Additionally, there are concerns about reductionist approaches to assessment in CBME, lack of good assessments for some competencies, and whether CBME frameworks include domains of current importance. This study will outline these issues and discuss the responses of the medical education community. METHODS: The concerns and challenges expressed are primarily categorised as: (i) those related to practical, administrative and logistical challenges in implementing CBME frameworks, and (ii) those with more conceptual or theoretical bases. The responses of the education community to these issues are then summarised. CONCLUSIONS: The education community has begun to address the challenges involved in implementing CBME. Models and guidance exist to inform implementation strategies across the continuum of education, and focus on the more efficient use of resources and technology, and the use of milestones and entrustable professional activities-based frameworks. Inconsistencies in CBME definitions and frameworks remain a significant obstacle. Evolution in assessment approaches from in vitro task-based methods to in vivo integrated approaches is responsive to many of the theoretical and conceptual concerns about CBME, but much work remains to be done to bring rigour and quality to work-based assessment.

Criteria for social media-based scholarship in health professions education.

BACKGROUND: Social media are increasingly used in health professions education. How can innovations and research that incorporate social media applications be adjudicated as scholarship? OBJECTIVE: To define the criteria for social media-based scholarship in health professions education. METHOD: In 2014 the International Conference on Residency Education hosted a consensus conference of health professions educators with expertise in social media. An expert working group drafted consensus statements based on a literature review. Draft consensus statements were posted on an open interactive online platform 2 weeks prior to the conference. In-person and virtual (via Twitter) participants modified, added or deleted draft consensus statements in an iterative fashion during a facilitated 2 h session. Final consensus statements were unanimously endorsed. RESULTS: A review of the literature demonstrated no existing criteria for social media-based scholarship. The consensus of 52 health professions educators from 20 organisations in four countries defined four key features of social media-based scholarship. It must (1) be original; (2) advance the field of health professions education by building on theory, research or best practice; (3) be archived and disseminated; and (4) provide the health professions education community with the ability to comment on and provide feedback in a transparent fashion that informs wider discussion. CONCLUSIONS: Not all social media activities meet the standard of education scholarship. This paper clarifies the criteria, championing social media-based scholarship as a legitimate academic activity in health professions education.

The transition to competency-based pediatric training in the United Arab Emirates.

Although competency-based medical education has become the standard for physician training in the West, many developing countries have not yet adopted competency-based training. In 2009 in the United Arab Emirates, the government regulatory and operational authorities for healthcare in Abu Dhabi mandated a wide-scale reform of the emirate's postgraduate residency programs to the competency-based framework of the newly formed Accreditation Council for Graduate Medical Education-International (ACGME-I). This article briefly describes the rationale for competency-based medical education and provides an overview of the transition from traditional, time-based residency training to competency-based postgraduate medical education for the Pediatrics residency programs in Abu Dhabi. We will provide data on the initial impact of this transition on resident performance and patient outcomes in a Pediatrics residency program in an academic medical center in the United Arab Emirates.