The Next Era of Assessment Within Medical Education: Exploring Intersections of Context and Implementation.

In competency-based medical education (CBME), which is being embraced globally, the patient-learner-educator encounter occurs in a highly complex context which contributes to a wide range of assessment outcomes. Current and historical barriers to considering context in assessment include the existing post-positivist epistemological stance that values objectivity and validity evidence over the variability introduced by context. This is most evident in standardized testing. While always critical to medical education the impact of context on assessment is becoming more pronounced as many aspects of training diversify. This diversity includes an expanding interest beyond individual trainee competence to include the interdependency and collective nature of clinical competence and the growing awareness that medical education needs to be co-produced among a wider group of stakeholders. In this Eye Opener, we wish to consider: 1) How might we best account for the influence of context in the clinical competence assessment of individuals in medical education? and by doing so, 2) How could we usher in the next era of assessment that improves our ability to meet the dynamic needs of society and all its stakeholders? The purpose of this Eye Opener is thus two-fold. First, we conceptualize - from a variety of viewpoints, how we might address context in assessment of competence at the level of the individual learner. Second, we present recommendations that address how to approach implementation of a more contextualized competence assessment.

Transforming remediation for competency-based medical education.

We care about the future experiences of all health professions trainees as competency-based medical education evolves. It is an exciting new era with many possibilities for progress in learning and competency development. Yet we are concerned that remediation remains a troubled and stigmatized detour from routine learning that can persist as a feared off-ramp from competency development rather than a central avenue for improvement and competency achievement. We believe that it is time to acknowledge that all trainees struggle and to recognize that remediation is an essential aspect of individualized learning. Decisive steps are possible to revitalize remediation and to launch its transformation towards growth-oriented pathways for change.

Validity evidence for the clinical communication skills assessment tool (CCSAT) from 9 years of implementation in a high stakes medical student OSCE.

OBJECTIVES: Communication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE. METHODS: We describe the implementation of the CCSAT including information on the underlying model, the tool's items, domains, scales and scoring, and its role in curriculum. Internal structure is explored through item, internal consistency, and confirmatory factor analyses. Evidence for CCSAT validity is synthesized within prevailing frameworks (Messick12 and Kane13) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research. RESULTS: Implementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications. CONCLUSIONS AND PRACTICE IMPLICATIONS: This pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.

Racial Implicit Bias and Communication Among Physicians in a Simulated Environment.

Importance: Racial implicit bias can contribute to health disparities through its negative influence on physician communication with Black patients. Interventions for physicians to address racial implicit bias in their clinical encounters are limited by a lack of high-fidelity (realistic) simulations to provide opportunities for skill development and practice. Objective: To describe the development and initial evaluation of a high-fidelity simulation of conditions under which physicians might be influenced by implicit racial bias. Design, Setting, and Participants: This cross-sectional study, performed on an online platform from March 1 to September 30, 2022, recruited a convenience sample of physician volunteers to pilot an educational simulation. Exposures: In the simulation exercise, physicians saw a 52-year-old male standardized patient (SP) (presenting as Black or White) seeking urgent care for epigastric pain, nausea, and vomiting. The case included cognitive stressors common to clinical environments, including clinical ambiguity, stress, time constraints, and interruptions. Physicians explained their diagnosis and treatment plan to the SP, wrote an assessment and management plan, completed surveys, and took the Race Implicit Association Test (IAT) and Race Medical Cooperativeness IAT. The SPs, blinded to the purpose of the study, assessed each physician's communication using skills checklists and global rating scales. Main Outcomes and Measures: Association between physicians' IAT scores and SP race with SP ratings of communication skills. Results: In 60 physicians (23 [38.3%] Asian, 4 [6.7%] Black, 23 [38.3%] White, and 10 [16.7%] other, including Latina/o/x, Middle Eastern, and multiracial; 31 [51.7%] female, 27 [45.0%] male, and 2 [3.3%] other), the interaction of physicians' Race IAT score and SP race was significant for overall communication (mean [SD] ß = -1.29 [0.41]), all subdomains of communication (mean [SD] ß = -1.17 [0.52] to -1.43 [0.59]), and overall global ratings (mean [SD] ß = -1.09 [0.39]). Black SPs rated physicians lower on communication skills for a given pro-White Race IAT score than White SPs; White SP ratings increased as physicians' pro-White bias increased. Conclusions and Relevance: In this cross-sectional study, a high-fidelity simulation calibrated with cognitive stressors common to clinical environments elicited the expected influence of racial implicit bias on physicians' communication skills. The outlined process and preliminary results can inform the development and evaluation of interventions that seek to address racial implicit bias in clinical encounters and improve physician communication with Black patients.

Librarian-Led Assessment of Medical Students' Evidence-Based Medicine Competency: Facilitators and Barriers.

Introduction: We must ensure, through rigorous assessment that physicians have the evidence-based medicine (EBM) skills to identify and apply the best available information to their clinical work. However, there is limited guidance on how to assess EBM competency. With a better understanding of their current role in EBM education, Health Sciences Librarians (HSLs), as experts, should be able to contribute to the assessment of medical student EBM competence. The purpose of this study is to explore the HSLs perspective on EBM assessment practices, both current state and potential future activities. Methods: We conducted focus groups with librarians from across the United States to explore their perceptions of assessing EBM competence in medical students. Participants had been trained to be raters of EBM competence as part of a novel Objective Structured Clinical Examination (OSCE). This OSCE was just the starting point and the discussion covered topics of current EBM assessment and possibility for expanded responsibilities at their own institutions. We used a reflexive thematic analysis approach to construct themes from our conversations. Results: We constructed eight themes in four broad categories that influence the success of librarians being able to engage in effective assessment of EBM: administrative, curricular, medical student, and librarian. Conclusion: Our results inform medical school leadership by pointing out the modifiable factors that enable librarians to be more engaged in conducting effective assessment. They highlight the need for novel tools, like EBM OSCEs, that can address multiple barriers and create opportunities for deeper integration of librarians into assessment processes.

Transformative Leadership Training in Medical Education: A Topology.

Issue: Efforts to improve medical education often focus on optimizing technical aspects of teaching and learning. However, without considering the connection between the pedagogical-curricular and the foundational philosophically-defined educational aims of medicine and medical education, critical system reform is unlikely. The transformation of medical education requires leaders uniquely prepared to view medicine and medical education critically as it is and as it ought to be, and who have the capacity to lead changes aimed at overcoming the identified gaps. This paper proposes a five-level topology to guide leaders to develop this capacity. Evidence: Without reference to a shared understanding of a larger, more profound philosophical vision of the ideal physician and of the educational process of "becoming" that physician, efforts to change medical education are likely to be incremental and insufficient rather than transformative. Such efforts may lead to frequent pedagogical-curricular reforms, shifting evaluation models, and paradigmatic conflicts in medical education systems across contexts. This paper describes a leadership program meant to develop transformational educational leaders. The leadership program is built on and teaches the five-level topology we describe here. The five levels are 1) Philosophy 2) Philosophy of Education 3) Theory of Practice 4) Implementation and 5) Evaluation. Implications: The leadership development program exemplifies how the topology can be implemented as a framework to foster transformation in medical education. The topology is a metaphor exemplified by the Mobius Strip, a continuous and never-broken object, which reflects the ways in which the five levels are inherently connected and reflect on each other. Medical education leadership requires deeper engagement with paradigmatic thought to transform the field for the future.

Understanding medical student paths to communication skills expertise using latent profile analysis.

PURPOSE: To describe patterns of clinical communication skills that inform curriculum enhancement and guide coaching of medical students. MATERIALS AND METHODS: Performance data from 1182 consenting third year medical students in 9 cohorts (2011-2019), on a 17-item Clinical Communication Skills Assessment Tool (CCSAT) completed by trained Standardized Patients as part of an eight case high stakes Comprehensive Clinical Skills Exam (CCSE) were analyzed using latent profile analysis (LPA). Assessment domains included: information gathering (6 items), relationship development (5 items), patient education (3 items), and organization/time management (3 items). LPA clustered learners with similar strength/weakness into profiles based on item response patterns across cases. One-way analysis of variance (ANOVA) assessed for significant differences by profile for CCSAT items. RESULTS: Student performance clustered into six profiles in three groups, high performing (HP1 and HP2-Low Patient Education, 15.7%), average performing (AP1 and AP2-Interrupters, 40.9%), and lower performing profiles (LP1-Non-interrupters and LP2, 43.4%) with adequate model fit estimations and similar distribution in each cohort. We identified 3 CCSAT items that discriminated among learner's skill profiles. CONCLUSION: Clinical communication skill performance profiles provide nuanced, benchmarked guidance for curriculum improvement and tailoring of communication skills coaching.

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine.

PURPOSE: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). METHOD: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified. RESULTS: 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations ( r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. CONCLUSIONS: For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.

Viewing Readiness-for-Residency through Binoculars: Mapping Competency-Based Assessments to the AAMC's 13 Core Entrustable Professional Activities (EPAs).

Construct: The construct being assessed is readiness-for-residency of graduating medical students, as measured through two assessment frameworks. Background: Readiness-for-residency of near-graduate medical students should be but is not consistently assessed. To address this, the Association of American Medical Colleges (AAMC), in 2014, identified and described 13 core Entrustable Professional Activities (EPAs), which are tasks that all residents should be able to perform unsupervised upon entering residency. However, the AAMC did not initially provide measurement guidelines or propose standardized assessments. We designed Night-onCall (NOC), an immersive simulation for our near-graduating medical students to assess and address their readiness-for-residency, framed around tasks suggested by the AAMC's core EPAs. In adopting this EPA assessment framework, we began by building upon an established program of competency-based clinical skills assessments, repurposing competency-based checklists to measure components of the EPAs where possible, and designing new checklists, when necessary. This resulted in a blended suite of 14 checklists, which theoretically provide substantive assessment of all 13 core EPAs. In this paper, we describe the consensus-based mapping process conducted to ensure we understood the relationship between competency and EPA-based assessment lenses and could therefore report meaningful feedback on both to transitioning students in the NOC exercise. Approach: Between January-November 2017, five clinician and two non-clinician health professions educators at NYU Grossman School of Medicine conducted a rigorous consensus-based mapping process, which included each rater mapping each of the 310 NOC competency-based checklist items to lists of entrustable behaviors expected of learners according to the AAMC 13 core EPAs. Findings: All EPAs were captured to varying degrees by the 14 NOC checklists (overall Intraclass Correlation Coefficient (ICC) = 0.77). Consensus meetings resolved discrepancies and improved ICC values for three (EPA-9, EPA-10, EPA-12) of the four EPAs that initially showed poor reliability. Conclusions: Findings suggest that with some limitations (e.g., EPA-7 "form clinical questions/retrieve evidence") established competency-based assessments can be repurposed to measure readiness-for-residency through an EPA lens and both can be reported to learners and faculty.

Creating Wikipedia articles on health and technology topics can empower the writers and benefit the community.

BACKGROUND: Publicly accessible information regarding imaging procedures is lacking, especially in non-English languages. Biomedical engineering students do not generally have opportunities to practice conveying scientific knowledge to the public. METHODS: As part of a Techniques and Clinical Usage of Medical Imaging Devices course, for extra credit, several biomedical engineering students choose to create and edit Wikipedia articles in the local language (Hebrew). The goal of this activity was to serve the local community, while improving students' abilities and self-perception in reading and reporting scientific knowledge. Following task completion, individual interviews were conducted with the students to assess the impact of the task on student personal development, sense of meaning and their view of their role in educating the public. RESULTS: Most students considered the task meaningful and impactful on society. Additional academic credit was not perceived as the most important incentive for participating. CONCLUSIONS: Medical and other professional schools should seek to include tasks such as writing Wikipedia articles in their curricula. Educational assignments that integrate academic work, student identity development and direct community benefit can have a long-term beneficial impact on learners and society.

REACH: A Required Curriculum to Foster the Well-Being of Medical Students.

PROBLEM: The extent of medical student unwellness is well documented. Learner distress may impact patient care, workforce adequacy, and learners' performance and personal health. The authors describe the philosophy, structure, and content of the novel REACH (Recognize, Empathize, Allow, Care, Hold each other up) curriculum and provide a preliminary evaluation. APPROACH: The REACH curriculum is a mandatory, longitudinal well-being curriculum for first- and second-year medical students at the Medical College of Wisconsin (MCW) designed to prepare them for the emotional life of being a physician. The curriculum uses a framework, core concepts, and skills from the field of trauma stewardship. It builds on effective medical student well-being interventions (e.g., mindfulness-based training) and the sharing of personal stories by instructors during didactic and small-group sessions that are integrated into the regular MCW curriculum. During the first 2 years of implementation (2018-2019 and 2019-2020), the curriculum was evaluated using mid- and postcurriculum student surveys. OUTCOMES: Over 700 students have completed the REACH curriculum as of March 2022. Overall, most students who responded to the surveys in 2018-2020 reported that they felt the REACH curriculum material was important, that the curriculum met their expectations for a quality medical school course, and that they would recommend other schools incorporate a similar curriculum. Respondents to the 2019-2020 postcurriculum survey indicated the REACH curriculum helped them develop self-care (84% [85/101]), mindfulness (76% [76/101]), and help-seeking (71% [72/101]) skills. NEXT STEPS: The initial outcomes show that integrating a mandatory well-being curriculum is feasible and acceptable to medical students. The authors plan to examine the relationships between student-reported well-being metrics, academic and clinical performance data, and professional identity formation. They are also prototyping electronic dashboards that will allow students to interact with their well-being data to promote timely help-seeking and behavior change.

A Preliminary Evaluation of Students' Learning and Performance Outcomes in an Accelerated 3-Year MD Pathway Program.

BACKGROUND: Little outcome data exist on 3-year MD (3YMD) programs to guide residency program directors (PDs) in deciding whether to select these graduates for their programs. OBJECTIVE: To compare performance outcomes of 3YMD and 4-year MD (4YMD) students at New York University Grossman School of Medicine. METHODS: In 2020, using the Kirkpatrick 4-level evaluation model, outcomes from 3 graduating cohorts of 3YMD students (2016-2018) were compared with the 4YMD counterparts. RESULTS: Descriptive statistics compared outcomes among consented student cohorts: 92% (49 of 53) 3YMD, 87% (399 of 459) 4YMD-G, and 84% (367 of 437) 4YMD-S. Student survey response rates were 93% (14 of 15), 74% (14 of 19), and 89% (17 of 19) from 2016 to 2018. PDs' response rates were 58% (31 of 53, 3YMD) and 51% (225 of 441, 4YMD). Besides age, 3YMD and 4YMD cohorts did not differ significantly in admissions variables. Other than small statistically significant differences in the medicine shelf examination (3YMD mean 74.67, SD 7.81 vs 4YMD-G mean 78.18, SD 7.60; t test=3.02; P=.003) and USMLE Step 1 (3YMD mean 235.13, SD 17.61 vs 4YMD-S mean 241.70, SD 15.92; t test=2.644; P=.009 and vs 4YMD-G mean 242.39, SD 15.65; t test=2.97; P=.003) and Step 2 CK scores (3YMD mean 242.57, SD 15.58 vs 4YMD-S mean 248.55, SD 15.33; t test=2.55; P=.01 and vs 4YMD-G mean 247.83, SD 15.38; t test=2.97; P=.03), other metrics and overall intern ratings did not differ by pathway. CONCLUSIONS: Exploratory findings from a single institution suggest that 3YMD students performed similarly to 4YMD students in medical school and the first year of residency.

Mentoring Underrepresented Minority Physician-Scientists to Success.

As the nation seeks to recruit and retain physician-scientists, gaps remain in understanding and addressing mitigatable challenges to the success of faculty from underrepresented minority (URM) backgrounds. The Doris Duke Charitable Foundation Fund to Retain Clinical Scientists program, implemented in 2015 at 10 academic medical centers in the United States, seeks to retain physician-scientists at risk of leaving science because of periods of extraordinary family caregiving needs, hardships that URM faculty-especially those who identify as female-are more likely to experience. At the annual Fund to Retain Clinical Scientists program directors conference in 2018, program directors-21% of whom identify as URM individuals and 13% as male-addressed issues that affect URM physician-scientists in particular. Key issues that threaten the retention of URM physician-scientists were identified through focused literature reviews; institutional environmental scans; and structured small- and large-group discussions with program directors, staff, and participants. These issues include bias and discrimination, personal wealth differential, the minority tax (i.e., service burdens placed on URM faculty who represent URM perspectives on committees and at conferences), lack of mentorship training, intersectionality and isolation, concerns about confirming stereotypes, and institutional-level factors. The authors present recommendations for how to create an environment in which URM physician-scientists can expect equitable opportunities to thrive, as institutions demonstrate proactive allyship and remove structural barriers to success. Recommendations include providing universal training to reduce interpersonal bias and discrimination, addressing the consequences of the personal wealth gap through financial counseling and benefits, measuring the service faculty members provide to the institution as advocates for URM faculty issues and compensating them appropriately, supporting URM faculty who wish to engage in national leadership programs, and sustaining institutional policies that address structural and interpersonal barriers to inclusive excellence.

Does a measure of Medical Professional Identity Formation predict communication skills performance?

OBJECTIVE: To validate an approach to measuring professional identity formation (PIF), we explore if the Professional Identity Essay (PIE), a stage score measure of medical professional identity (PI), predicts clinical communication skills. METHODS: Students completed the PIE during medical school orientation and a 3-case Objective Structured Clinical Exam (OSCE) where standardized patients reliably assessed communication skills in 5 domains. Using mediation analyses, relationships between PIE stage scores and communication skills were explored. RESULTS: For the 351 (89%) consenting students, controlling for individual characteristics, there were increases in patient counseling (6.5%, p<0.01), information gathering (4.3%, p = 0.01), organization and management (4.1%, p = 0.02), patient assessment (3.6%, p = 0.04), and relationship development (3.5%, p = 0.03) skills for every half stage increase in PIE score. The communication skills of lower socio-economic status (SES) students are indirectly impacted by their slightly higher PIE stage scores. CONCLUSION: Higher PIE stage scores are associated with higher communication skills and lower SES. PRACTICE IMPLICATIONS: PIE predicts critical clinical skills and identifies how SES and other characteristics indirectly impact future clinical performance, providing validity evidence for using PIE as a tool in longitudinal formative academic coaching, program and curriculum evaluation, and research.

Telehealth and Medical Student Education in the Time of COVID-19-and Beyond.

The COVID-19 pandemic continues to limit medical students' full reintegration into clinical learning environments, thus exacerbating an ongoing challenge in identifying a robust number of clinical educational activities at excellent clinical sites for all students. Because medical students across the United States were removed from direct patient care activities in mid-March 2020 due to COVID-19, medical centers have prioritized and implemented changes to the process of patient care. As some barriers are being lifted in the face of a highly contagious and deadly infection, the use of telehealth (delivery of health services remotely via telephone, video, and secure messaging), although not new, is rapidly expanding into all aspects of patient care. Health care providers have been encouraged to conduct many interactions at a physical distance. Telehealth largely replaced face-to-face visits for nonemergency care in an attempt to slow viral transmission while enabling physicians to continue to deliver patient education, manage acute and chronic illness, and nurture caring doctor-patient relationships. Health care providers, many of whom were initially reluctant to embrace telehealth technology and logistics, are becoming nimbler and more aware of the many positive aspects of telehealth. The authors suggest that integrating medical students into telehealth activities would help maintain and improve patients' health, extend the capabilities of health care teams and systems during and after the pandemic, and increase medical students' opportunities for experiential learning and professional identity formation. The authors expand on these 3 goals, suggest several concrete student telehealth activities, propose a curricular strategy, and outline opportunities to overcome key barriers to full alignment of telehealth and undergraduate medical education.

In Search of Medical Professionalism Research: Preliminary Results from a Review of Widely Read Medical Journals.

INTRODUCTION: Professionalism is a core concept in medicine. The extent to which knowledge about professionalism is anchored in empirical research is unknown. Understanding the current state of research is necessary to identify significant gaps and create a road map for future professionalism efforts. The authors conducted an exploratory literature review to characterize professionalism research published in widely read medical journals, identify knowledge gaps, and describe the sources of funding for the identified studies. METHODS: The authors focused on Medline's Abridged Index Medicus and 4 core Medline education-oriented journal and developed a search filter using text words found in the article title or abstract addressing professionalism. Articles were further filtered to include those indicating a research focus. RESULTS: The search strategy resulted in 461 professionalism research articles for analysis. Articles were divided into themes of education (n = 212, 45.9%), performance (n = 83, 18%), measurement development (n = 13, 2.8%), remediation (n = 53, 11.5%), and well-being (n = 100, 21.6%). There were 36 studies from 1980 to 2002 (Era 1: before publication of Accreditation Council for Graduate Medical Education competencies) and 425 from 2003 to 17 (Era 2: after Accreditation Council for Graduate Medical Education publication of competencies). Professionalism education was the most common topic area, and most studies were from single institutions with results based on convenience samples. Most studies received no funding or were funded by the authors' own institution. DISCUSSION: Little empirical research is available on professionalism in widely read medical journals. There has been limited external research funding available to study this topic. CONCLUSION: More investment in high quality professionalism research is justified and should be encouraged.

Graduate level health professions education: how do previous work experiences influence perspectives about interprofessional collaboration?

Understanding how previous experiences with interprofessional education and collaboration inform health care provider perspectives is important for developing interprofessional interventions at the graduate level. The purpose of this study was to examine how previous work experiences of graduate level health professions students inform perspectives about interprofessional education and collaboration. Drawing from program evaluation data of two separate graduate level interprofessional education interventions based in primary care and home health care, we conducted a qualitative secondary data analysis of 75 interviews generated by focus groups and individual interviews with graduate students from 4 health professions cadres. Using directed content analysis, the team coded to capture descriptions of interprofessional education or collaboration generated from participants' previous work experiences. Coding revealed 173 discrete descriptions related to previous experiences of interprofessional education or collaboration. Three themes were identified from the analysis that informed participant perspectives: Previous educational experiences (including work-based training); previous work experiences; and organizational factors and interprofessional collaboration. Experiences varied little between professions except when aspects of professional training created unique circumstances. The study reveals important differences between graduate and undergraduate learners in health professions programs that can inform interprofessional education and collaboration intervention design.

Development and maintenance of a medical education research registry.

BACKGROUND: Medical Education research suffers from several methodological limitations including too many single institution, small sample-sized studies, limited access to quality data, and insufficient institutional support. Increasing calls for medical education outcome data and quality improvement research have highlighted a critical need for uniformly clean and easily accessible data. Research registries may fill this gap. In 2006, the Research on Medical Education Outcomes (ROMEO) unit of the Program for Medical Innovations and Research (PrMEIR) at New York University's (NYU) Robert I. Grossman School of Medicine established the Database for Research on Academic Medicine (DREAM). DREAM is a database of routinely collected, de-identified undergraduate (UME, medical school leading up to the Medical Doctor degree) and graduate medical education (GME, residency also known as post graduate education leading to eligibility for specialty board certification) outcomes data available, through application, to researchers. Learners are added to our database through annual consent sessions conducted at the start of educational training. Based on experience, we describe our methods in creating and maintaining DREAM to serve as a guide for institutions looking to build a new or scale up their medical education registry. RESULTS: At present, our UME and GME registries have consent rates of 90% (n = 1438/1598) and 76% (n = 1988/2627), respectively, with a combined rate of 81% (n = 3426/4225). 7% (n = 250/3426) of these learners completed both medical school and residency at our institution. DREAM has yielded a total of 61 individual studies conducted by medical education researchers and a total of 45 academic journal publications. CONCLUSION: We have built a community of practice through the building of DREAM and hope, by persisting in this work the full potential of this tool and the community will be realized. While researchers with access to the registry have focused primarily on curricular/ program evaluation, learner competency assessment, and measure validation, we hope to expand the output of the registry to include patient outcomes by linking learner educational and clinical performance across the UME-GME continuum and into independent practice. Future publications will reflect our efforts in reaching this goal and will highlight the long-term impact of our collaborative work.