Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice.

BACKGROUND: As a community of practice (CoP), medical education depends on its research literature to communicate new knowledge, examine alternative perspectives, and share methodological innovations. As a key route of communication, the medical education CoP must be concerned about the rigor and validity of its research literature, but prior studies have suggested the need to improve medical education research quality. Of concern in the present study is the question of how responsive the medical education research literature is to changes in the CoP. We examine the nature and extent of changes in the quality of medical education research over a decade, using a widely cited study of research quality in the medical education research literature as a benchmark to compare more recent quality indicators. METHODS: A bibliometric analysis was conducted to examine the methodologic quality of quantitative medical education research studies published in 13 selected journals from September 2013 to December 2014. Quality scores were calculated for 482 medical education studies using a 10-item Medical Education Research Study Quality Instrument (MERSQI) that has demonstrated strong validity evidence. These data were compared with data from the original study for the same journals in the period September 2002 to December 2003. Eleven investigators representing 6 academic medical centers reviewed and scored the research studies that met inclusion and exclusion criteria. Primary outcome measures include MERSQI quality indicators for 6 domains: study design, sampling, type of data, validity, data analysis, and outcomes. RESULTS: There were statistically significant improvements in four sub-domain measures: study design, type of data, validity and outcomes. There were no changes in sampling quality or the appropriateness of data analysis methods. There was a small but significant increase in the use of patient outcomes in these studies. CONCLUSIONS: Overall, we judge this as equivocal evidence for the responsiveness of the research literature to changes in the medical education CoP. This study identified areas of strength as well as opportunities for continued development of medical education research.

Personality Compatibility Within Faculty Mentoring Dyads and Perceived Mentoring Outcomes: Survey Results of Academic Medicine Institutions in the USA.

Mentorship is recognized as a critical approach to support successful careers in academic medicine. Obstacles to successful mentoring relationships include difficulty finding appropriate mentors and poor alignment of mentee/mentor interests and goals. We set out to investigate if concordance or discordance in personality traits of mentees and mentors impacts perceived success of mentoring relationships. Our findings indicated that concordance and discordance on one personality trait, neuroticism, seemed to significantly impact the perceived mentoring relationship success related to career progression. Further work to refine criteria for matching mentees and mentors considering personality traits may impact the effectiveness of mentoring relationships.

Influences on and Characteristics of the Professional Identity Formation of Clinician Educators: A Qualitative Analysis.

PURPOSE: Professional identity formation is the process of internalizing the ideals, values, and beliefs of a profession. In recent years, research on clinician-educator (CE) identity formation has expanded, yet gaps exist in understanding initial influences on an educator identity, sustainment throughout a career, and development of successful pathways for early CEs. This study explored the initial influences on and characteristics of the professional identity formation of CEs in an age-diverse, multispecialty population in the United States. METHOD: This was a cross-sectional qualitative study of a purposive sample of medical educators at 6 institutions across the United States between 2018 and 2019. Focus groups were conducted to obtain participants' perspectives on their career choice and subsequent formation of their professional identity as CEs. The authors used a thematic analysis of focus group data to identify themes and domains through an iterative process. RESULTS: Twelve focus groups were conducted with a total of 93 participants. Responses were categorized into 5 domains: community supportive of medical education, culture of institution and training, personal characteristics, facilitators, and professionalization of medical education. Themes highlighted the importance of role models and mentors, an affinity and aptitude for teaching and education, specific facilitators for entry into a career in medical education, the evolution from a layperson, importance of formalized training programs, and a supportive academic community. CONCLUSIONS: Clinicians experienced a variety of factors that influenced their initial career choice in medical education and subsequent professional identity formation as a CE. This study confirms and expands the current understanding of this process in an age-diverse, multispecialty population of CEs. Educators and administrators designing career development programs across the continuum of medical education should consider these aspects as they mentor and support their learners and faculty.

A Perfect Match? A Scoping Review of the Influence of Personality Matching on Adult Mentoring Relationships-Implications for Academic Medicine.

INTRODUCTION: Mentoring is a widely regarded faculty development strategy in academic medicine. However, the lack of understanding about mentoring relationship dynamics limits effective recruitment, implementation, and evaluation. Despite decades of publications describing adult mentoring initiatives, few studies examine personality influence in mentoring relationships. This scoping review examined the extent, range, and nature of the research on personality matching in mentoring relationships, and identified research gaps in the literature. METHODS: Scoping review methodology guided a search of six databases representing higher education, health sciences education, and professional contexts where mentoring is used. Consistent with the inclusive approach of a scoping review, authors included academic papers and other article types. RESULTS: The scoping review yielded 39 articles. Literature mostly originated in the United States, publication sources represented multiple disciplines, and the context for the majority of articles was the workplace. The most common publication type was a research report. Although all articles addressed personality or mentoring, only three articles examined personality matching and its contribution to the mentoring relationship. Finally, although the Big Five personality traits were cited in multiple studies, other personality frameworks were used. DISCUSSION: Academic medicine expends resources developing and supporting mentoring programs but there remains limited understanding of how best to identify and match mentors and protégés. Further understanding of the role of joint and unique personality traits in academic medicine mentoring relationships seems necessary, if the field continues to invest, time, money, and resources for mentoring programs.

Applying multiple frameworks to establish effective virtual collaborative teams in academia: a review and recommendations.

Recently, academic health professionals have been increasing collaboration with peers at a distance for activities such as research, scholarship, and faculty development. Novel virtual technologies enable academic professional teams to overcome time and distance barriers to facilitate collaboration, but little research is available to guide academicians on how to effectively organize and manage virtual collaborative teams using these technologies. Based upon a literature review and six years of experience as a virtual collaborative team, the authors use Boyer's Scholarship of Integration paradigm to identify and critique four models for virtual collaboration. The literature search devised from the four identified models found references that had a theoretical foundation for peer virtual collaboration and have been adopted in some professional context. The authors present a review of this literature, describe the benefits for adapting these models to academic health profession contexts, and provide a reflective critique about the challenges for their adaptation in these contexts. They also provide a hypothetical scenario to exemplify the application of these models for health-care professionals along with important considerations and tips when forming new virtual peer collaborative teams or problem-solving teams who are not optimally functioning.

The CARE Inventory: A Self-Reflective, Behavior-Based Instrument to Guide Professional Development and Mentorship of Academic Faculty.

Introduction: Traditional needs assessments often rely on self-reported skill levels. To gather more objective and growth-focused data, we developed a behavior-based inventory to measure perceived faculty competence and desired areas for growth in four common domains of academic medicine: clinical, administrative/leadership, research, and education (CARE). Methods: Competencies in teaching, research, and professional development and leadership noted in the literature were used as the foundation of our instrument. Clinical service topics were added, and questions were vetted with the executive committee of our faculty development center. A behavior-based inventory was developed to enhance self-reporting of competency. The instrument was piloted with faculty at an external institution and revisions made prior to data collection. Results: In May 2016, the CARE Inventory was sent to all full-time faculty members in the Ohio State University College of Medicine (n = ∼1,800). We received 350 responses (20% response rate). Individual reports were generated and sent to the faculty member and his/her identified mentor for individual professional development. Summary data were used in aggregate for professional development program planning. Discussion: Anchoring measurement to current and desired future behavior allows for more self-reflective and growth-oriented assessment for individuals, and results can inform tactical faculty development by both individual and program. We believe this is a scalable and generalizable instrument other academic medical and health sciences programs could use both as a needs assessment tool for program planning and for individualized development plans with faculty and their mentors.

Milestone-Based Tool for Learner Evaluation of Faculty Clinical Teaching.

Introduction: Traditional normative Likert-type evaluations of faculty teaching have several drawbacks, including lack of granular feedback, potential for inflation, and the halo effect. To provide more meaningful data to faculty on their teaching skills and encourage educator self-reflection and skill development, we designed and implemented a milestone-based faculty clinical teaching evaluation tool. Methods: The evaluation tool contains 10 questions that assess clinical teaching skills with descriptive milestone behavior anchors. Nine of these items are based on the Stanford Faculty Development Clinical Teaching Model and annual Accreditation Council for Graduate Medical Education (ACGME) resident survey questions; the tenth was developed to address professionalism at our institution. The tool was developed with input from residency program leaders, residents, and the faculty development committee and piloted with graduate medical education learners before implementation. Results: More than 7,200 faculty evaluations by learners and 550 faculty self-evaluations have been collected. Learners found the form easy to use and preferred it to previous Likert-based evaluations. Over the 2 years that faculty self-evaluations have been collected, their scores have been similar to the learner evaluation scores. The feedback provided faculty with more meaningful data on teaching skills and opportunities for reflection and skill improvement and was used in constructing faculty teaching skills programs at the institutional level. Discussion: This innovation provides an opportunity to give faculty members more meaningful teaching evaluations and feedback. It should be easy for other institutions and programs to implement. It leverages a familiar milestone construct and incorporates important ACGME annual resident survey information.

The Flipped Classroom in Medical Education: Engaging Students to Build Competency.

The flipped classroom represents an essential component in curricular reform. Technological advances enabling asynchronous and distributed learning are facilitating the movement to a competency-based paradigm in healthcare education. At its most basic level, flipping the classroom is the practice of assigning students didactic material, traditionally covered in lectures, to be learned before class while using face-to-face time for more engaging and active learning strategies. The development of more complex learning systems is creating new opportunities for learning across the continuum of medical education as well as interprofessional education. As medical educators engage in the process of successfully flipping a lecture, they gain new teaching perspectives, which are foundational to effectively engage in curricular reform. The purpose of this article is to build a pedagogical and technological understanding of the flipped classroom framework and to articulate strategies for implementing it in medical education to build competency.

Competency-based curricular design to encourage significant learning.

Most significant learning (SL) experiences produce long-lasting learning experiences that meaningfully change the learner's thinking, feeling, and/or behavior. Most significant teaching experiences involve strong connections with the learner and recognition that the learner felt changed by the teaching effort. L. Dee Fink in Creating Significant Learning Experiences: An Integrated Approach to Designing College Course defines six kinds of learning goals: Foundational Knowledge, Application, Integration, Human Dimension, Caring, and Learning to Learn. SL occurs when learning experiences promote interaction between the different kinds of goals, for example, acquiring knowledge alone is not enough, but when paired with a learning experience, such as an effective patient experience as in Caring, then significant (and lasting) learning occurs. To promote SL, backward design principles that start with clearly defined learning goals and the context of the situation of the learner are particularly effective. Emphasis on defining assessment methods prior to developing teaching/learning activities is the key: this ensures that assessment (where the learner should be at the end of the educational activity/process) drives instruction and that assessment and learning/instruction are tightly linked so that assessment measures a defined outcome (competency) of the learner. Employing backward design and the AAMC's MedBiquitous standard vocabulary for medical education can help to ensure that curricular design and redesign efforts effectively enhance educational program quality and efficacy, leading to improved patient care. Such methods can promote successful careers in health care for learners through development of self-directed learning skills and active learning, in ways that help learners become fully committed to lifelong learning and continuous professional development.

Applying adult learning practices in medical education.

The application of the best practices of teaching adults to the education of adults in medical education settings is important in the process of transforming learners to become and remain effective physicians. Medical education at all levels should be designed to equip physicians with the knowledge, clinical skills, and professionalism that are required to deliver quality patient care. The ultimate outcome is the health of the patient and the health status of the society. In the translational science of medical education, improved patient outcomes linked directly to educational events are the ultimate goal and are best defined by rigorous medical education research efforts. To best develop faculty, the same principles of adult education and teaching adults apply. In a systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education, the use of experiential learning, feedback, effective relationships with peers, and diverse educational methods were found to be most important in the success of these programs. In this article, we present 5 examples of applying the best practices in teaching adults and utilizing the emerging understanding of the neurobiology of learning in teaching students, trainees, and practitioners. These include (1) use of standardized patients to develop communication skills, (2) use of online quizzes to assess knowledge and aid self-directed learning, (3) use of practice sessions and video clips to enhance significant learning of teaching skills, (4) use of case-based discussions to develop professionalism concepts and skills, and (5) use of the American Academy of Pediatrics PediaLink as a model for individualized learner-directed online learning. These examples highlight how experiential leaning, providing valuable feedback, opportunities for practice, and stimulation of self-directed learning can be utilized as medical education continues its dynamic transformation in the years ahead.

To play or not to play: leveraging video in medical education.

While video is a powerful teaching and learning tool because it can influence knowledge, skills, and attitude formation effectively and reach learners with various learning and communication styles, there are pedagogical, technical, and copyright considerations. Instructors must know sources of appropriate videos, select effective video segments, apply various strategies for incorporating video triggers into the overall educational process, refine the message, overcome technological obstacles, and comply with copyright laws. One might ask, "Is using video triggers to improve your teaching worth it?" "Yes!" Numerous studies demonstrate that using video in many medical education settings supports and enhances learning and offers a bigger advantage in contrast with traditional methods.

Learning management systems: technology to measure the medical knowledge competency of the ACGME.

AIMS: We report how the learning management system (LMS) Web Course Tools (WebCT) was used to design, implement and evaluate the web-based course "Principles of Ambulatory Paediatrics", taken by paediatric residents during an ambulatory block rotation. This report also illustrates how WebCT can be used to measure the medical knowledge competency required by the Accreditation Council for Graduate Medical Education (ACGME). METHODS: Eighty paediatric residents completed a 1-month outpatient rotation between July 1, 2001 and June 30, 2002. During this rotation residents were required to complete 4 modules in asthma, otitis media, gastroenteritis and fever, respectively. Each module was evaluated using a standard questionnaire. RESULTS: Completion rates for the required modules ranged from 64-72%. Residents in all 3 years of training showed improvement between the pre- and post-test scores for each module, except for postgraduate Year 2 residents in the asthma module. Most residents somewhat agreed, agreed or strongly agreed that the module components were useful and that the experience of completing the modules would improve their ability to take care of patients. CONCLUSIONS: The LMS WebCT is an innovative and adaptable approach for designing a web-based course for primary care education in paediatrics. The LMS addresses the educational needs of both a clinical division and a residency programme. The LMS also provides an information technology infrastructure to measure the medical knowledge competency required by the ACGME.