Stepwise Simulation Course Design Model: Survey Results from 16 Centers.

BACKGROUND AND OBJECTIVES: In 2016 we published a stepwise evidence-based model (subsequently named SimSteps) for curriculum development (CD) of simulation-based courses. The current study aimed to assess the uses, user friendliness, and perceived effectiveness of this model and its worksheet and to obtain suggestions for improvement. METHODS: We sent e-mail invitations for a 14-question web-based survey to 13 health professionals who requested the supplemental worksheet of the stepwise model and 11 authors who cited the model's publication in 14 articles. The survey included quantitative and qualitative items. RESULTS: Sixteen (67%) from seven countries and six professions responded. Ten (63%) used the model: six for both course and faculty development, three for course development only, and one for faculty development only. Both users and nonusers found the model and worksheet applicable and user friendly and agreed that they guided use of a systematic, comprehensive approach to CD. 94% (15 of 16) agreed that they helped CDers integrate educational effectiveness criteria, develop more objective learners' assessment tools, and enhance validity for their courses. Sixty-nine percent (11 of 16) agreed that model and its worksheet helped CDers include nontechnical skills in courses. The highest reported role in enhancing program evaluation results was in the gain of knowledge (five of eight, 63%) and least was clinical outcomes (two of eight, 25%). All respondents would recommend the model and worksheet to a colleague. CONCLUSION: Respondents find the stepwise model and its worksheet user friendly and helpful in developing simulation curricula of high educational standards. Future studies should include larger sample size, objective measures of impact, and longer-term follow-up.

Narrative Approach to Goals of Care Discussions: A Novel Curriculum.

CONTEXT: Innovative patient-centered approaches to goals of care (GOC) communication training are needed. Teaching a narrative approach, centered on the patient's unique story, is conceptually sound but has not been evaluated with respect to objective skills attainment. We developed a curriculum based on a novel, easily-remembered narrative approach to GOC, the 3-Act Model, and piloted it with a cohort of internal medicine (IM) interns. OBJECTIVES: To describe the development of the 3-Act Model curriculum and to assess its impact on the GOC communication skills of IM interns. METHODS: The curriculum was developed with input from multidisciplinary experts, IM residents, and patient/family representative. Notable elements included instrument development with validity evidence established, determination of proficiency standards, and creation of role-play scenarios. In two three-hour workshops, interns participated in role-plays as both providers and patients, before and after teaching (which included narrative reflection, didactics, and video demonstration). RESULTS: 22 interns played the role of provider in five unique scenarios; 106 proficiency ratings were analyzable. Interns objectively rated as proficient increased from 30% (pretest) to 100% (final role-play). By the end of the training, 96% of interns strongly agreed or agreed that they felt ready to independently lead basic GOC discussions and the percentage who strongly agreed increased with successive role-plays. All interns indicated they would recommend the training. CONCLUSION: This pilot demonstrates that the 3-Act Model is teachable and appreciated by learners. This GOC curriculum is the first based on a narrative approach to demonstrate objective skills improvement.

From Modules to MOOCs: Application of the Six-Step Approach to Online Curriculum Development for Medical Education.

Online curricula can make high-quality health professions education accessible in virtually any setting. They can enhance teaching and learning by both standardizing curricular resources and individualizing curricular experiences. Despite growing demand for and institutional interest in online curricula for medical education, many medical educators lack a framework for online curriculum development. Without rigorous and thoughtful development, online curricula can waste opportunity and resources by leading to education that is inferior to traditional methods. In this article, the authors describe a systematic approach to online curriculum development based on the Six-Step Approach for Curriculum Development for Medical Education, a widely used method that has led to successful implementation of a variety of traditional and online curricula. In each step, special considerations for curricula with larger and more diverse learner audiences-characteristic of many online curricula-are highlighted. Four common online curricular formats are also discussed: blended curricula, instructor-led fully online curricula, self-paced modules, and massive open online courses (MOOCs). The authors emphasize factors that differentiate one online format from another, including the budgetary, technical, and human resource requirements for each. The article concludes by urging medical educators to pursue opportunities to study and disseminate online curricular work.

How to identify and prioritize procedures suitable for simulation-based training: Experiences from general needs assessments using a modified Delphi method and a needs assessment formula.

Simulation-based training (SBT) is a revolutionary addition to the education of residents in different specialties. However, the selection of training programs is often reliant on commercially available equipment or on experiential notions, local interests or coincidence. The first step to developing training programs should be problem identification and general needs assessment to ensure that these are aligned with current trainee needs. We describe a stepwise approach to performing a needs assessment following the Delphi process and using a needs assessment formula (NAF) to prioritize technical procedures for simulation training. This technique relies on a panel of experts to gather information and achieve a convergence of opinion. We used three rounds of survey iterations. Round 1: A brainstorming phase to gather all procedures that a newly qualified resident should be able to perform. Round 2: Rating and preliminary prioritization with a NAF including frequency of the procedure, number of doctors, impact, and feasibility. Round 3: Elimination and final prioritization of procedures. The needs assessment identifies and prioritizes procedures for simulation training. It can guide medical educators in the planning of SBT programs to address current trainee needs with the aim of improving the education and training of future doctors.

Clinical coaching: Evolving the apprenticeship model for modern housestaff.

Feedback is one of the core components of teaching in the clinical setting. Traditionally, this activity has emphasized observations made by senior physicians and delivered to medical trainees. However, the optimal approach to feedback remains uncertain, and the literature abounds with trainee-perceived inadequacies in feedback content, quality, and impact. Moreover, given the multiplicity of demands on trainees and their physician mentors, we propose that medical trainees themselves-specifically, medical residents-are poised to serve as unique adjunct effectors of feedback. We propose a model of "clinical coaching" for residents as teachers, with emphasis on the active roles of both the feedback "giver" and "recipient". We define "clinical coaching" as "a helping longitudinal relationship between coach and apprentice that provides continuing feedback on and assistance with improving performance." Here, "coach" is the more experienced trainee (e.g. supervising resident), and "apprentice" is the less experienced trainee (e.g. intern or medical student). By working to better recognize and prepare residents for this vital role, we propose to encourage efforts to optimize the structure, execution, and impact of feedback in the contemporary climate of medical education.

A stepwise model for simulation-based curriculum development for clinical skills, a modification of the six-step approach.

BACKGROUND: Despite the rapid growth in the use of simulation in health professions education, courses vary considerably in quality. Many do not integrate efficiently into an overall school/program curriculum or conform to academic accreditation requirements. Moreover, some of the guidelines for simulation design are specialty specific. STUDY DESIGN: We designed a model that integrates best practices for effective simulation-based training and a modification of Kern et al.'s 6-step approach for curriculum development. We invited international simulation and health professions education experts to complete a questionnaire evaluating the model. We reviewed comments and suggested modifications from respondents and reached consensus on a revised version of the model. RESULTS: We recruited 17 simulation and education experts. They expressed a consensus on the seven proposed curricular steps: problem identification and general needs assessment, targeted needs assessment, goals and objectives, educational strategies, individual assessment/feedback, program evaluation, and implementation. We received several suggestions for descriptors that applied the steps to simulation, leading to some revisions in the model. CONCLUSION: We have developed a model that integrates principles of curriculum development and simulation design that is applicable across specialties. Its use could lead to high-quality simulation courses that integrate efficiently into an overall curriculum.

Qualitative synthesis and systematic review of otolaryngology in undergraduate medical education.

OBJECTIVE: Although 25% of primary care complaints are otolaryngology related, otolaryngology instruction is not required in most medical schools. Our aim was to systematically review existing literature on the inclusion of otolaryngology in undergraduate medical education. DATA SOURCES: PubMed, Embase, Cumulative Index to Nursing and Allied Health Literature, and Education Resources Information Center. STUDY DESIGN/REVIEW METHODS: Our search encompassed all indexed years through December 29, 2014. Inclusion criteria were English language, original human data, and a focus on medical student education. Data regarding study design, teacher, educational topic, educational methods, and setting were extracted from each article. Two investigators independently reviewed all articles. RESULTS: Our initial search yielded 436 articles; 87 underwent full-text evaluation and 47 remained in the final review. The majority of studies were conducted in the United States (40%), United Kingdom (23%), and Canada (17%) and represented a single institutional experience. Studies were classified as needs assessments (36%), curriculum descriptions (15%), educational methods (36%), and skills assessments (32%); 81% were levels of evidence 3 or 4. Most reports indicated that otolaryngology rotations are not compulsory. CONCLUSIONS: Studies indicated the need for increased exposure to otolaryngology. Educational methods such as team-based learning, simulation, online learning, and clinical skills assessments may offer ways to increase exposure without overburdening clinical faculty and require further study. Data suggest that a universal otolaryngology medical student curriculum would be valuable and aid in resource sharing across institutions. We recommend that an assessment be performed to determine topics and skills that should comprise this curriculum. LEVEL OF EVIDENCE: NA.

Preparing culture change agents for academic medicine in a multi-institutional consortium: the C - change learning action network.

INTRODUCTION: Research suggests an ongoing need for change in the culture of academic medicine. This article describes the structure, activities and evaluation of a culture change project: the C - Change Learning Action Network (LAN) and its impact on participants. The LAN was developed to create the experience of a culture that would prepare participants to facilitate a culture in academic medicine that would be more collaborative, inclusive, relational, and that supports the humanity and vitality of faculty. METHODS: Purposefully diverse faculty, leaders, and deans from 5 US medical schools convened in 2 1/2-day meetings biannually over 4 years. LAN meetings employed experiential, cognitive, and affective learning modes; innovative dialogue strategies; and reflective practice aimed at facilitating deep dialogue, relationship formation, collaboration, authenticity, and transformative learning to help members experience the desired culture. Robust aggregated qualitative and quantitative data collected from the 5 schools were used to inform and stimulate culture-change plans. RESULTS: Quantitative and qualitative evaluation methods were used. Participants indicated that a safe, supportive, inclusive, collaborative culture was established in LAN and highly valued. LAN members reported a deepened understanding of organizational change, new and valued interpersonal connections, increased motivation and resilience, new skills and approaches, increased self-awareness and personal growth, emotional connection to the issues of diversity and inclusion, and application of new learnings in their work. DISCUSSION: A carefully designed multi-institutional learning community can transform the way participants experience and view institutional culture. It can motivate and prepare them to be change agents in their own institutions.

Changing the culture of academic medicine: the C-Change learning action network and its impact at participating medical schools.

The culture of academic medicine has been described as hierarchical, competitive, and not highly supportive of female or minority faculty. In response to this, the authors designed the Learning Action Network (LAN), which was part of the National Initiative on Gender, Culture and Leadership in Medicine (C-Change). The LAN is a five-school consortium aimed at changing the organizational culture of its constituent institutions. The authors selected LAN schools to be geographically diverse and representative of U.S. medical schools. Institutional leaders and faculty representatives from constituent schools met twice yearly for four years (2006-2010), forming a cross-institutional learning community. Through their quarterly listing of institutional activities, schools reported a wide array of actions. Most common were increased faculty development and/or mentoring, new approaches to communication, and adoption of new policies and procedures. Other categories included data collection/management, engagement of key stakeholders, education regarding gender/diversity, and new/expanded leadership positions. Through exit interviews, most participants reported feeling optimistic about maintaining the momentum of change. However, some, especially in schools with leadership changes, expressed uncertainty. Participants reported that they felt that the LAN enabled, empowered, facilitated, and/or caused the reported actions.For others who might want to work toward changing the culture of academic medicine, the authors offer several lessons learned from their experiences with C-Change. Most notably, people, structures, policies, and reward systems must be put into place to support cultural values, and broad-based support should be created in order for changes to persist when inevitable transitions in leadership occur.

Defiance, compliance, or alliance? How we developed a medical professionalism curriculum that deliberately connects to cultural context.

BACKGROUND: In the age of globalization, non-Western medical educators seem too eager to conform to Western educational approaches and may, thereby, undermine the pursuit of local curricular needs. AIMS: To develop a medical professionalism curriculum that explicitly considered local cultural needs and social expectations. METHOD: We used a systematic six-step approach to develop the curriculum. RESULTS: We engaged local stakeholders (physicians, allied health professionals, and members of the public) in a nominal group process to identify professionalism competencies. Students and faculty participated in a survey and/or focus groups to determine learner/faculty needs. Teachers drafted goals and objectives related to locally valued competencies. We designed and implemented educational strategies to develop students' competencies that meet local societal expectations, such as involving family members in decision making. We plan to use multi-source feedback and a portfolio to assess students, which reinforces a definition of integrity that encompasses not only congruence between individual values and behaviors, but also achieving harmony among all stakeholders. We plan to reinforce the formal curriculum with faculty development and attention to the hidden curriculum. CONCLUSIONS: Based upon our experience and reflection, we offer some practical methods for integrating local cultural values and societal needs in professionalism education.

Teaching and assessing technical proficiency in surgical subspecialty fellowships.

OBJECTIVES: To determine how programs are teaching and assessing procedural skills, and their perceived success. DESIGN: Cross-sectional survey. SETTING: Accreditation Council for Graduate Medical Education (ACGME) approved training programs in pediatric urology and colorectal surgery. PARTICIPANTS: Program directors and recent graduates (2007-2009). RESULTS: Thirty-nine program directors (60%), and 57 graduates (64%) responded; 89.5% of graduates and 94.9% of program directors felt training occurred successfully for the procedures that trainees were performing in their present practice. Nearly 90% of trainees and all program directors reported that there was no formal assessment of procedural competency at the beginning of training, although 66.7% of program directors reported that trainees were assessed "informally." Both program directors and trainees reported dialogue with faculty was the most frequent method used in preparing for operative procedures. Other methods (textbook/atlas, journals, web-based programs, videos) were used less frequently. Program directors with shorter tenure were more likely to use web-based and video methods; younger trainees were less likely to use textbooks/atlases. Faculty feedback on clinical decision-making and postprocedural review were perceived by both program directors and trainees as the most effective assessment methods for improving performance; however, trainees were more likely than program directors to report that postprocedure reviews were not included in their training (15.8% vs 9%, p = 0.045). Patient outcomes, written feedback from peers, and self-assessment were included in most programs, but valued less. Simulation was used in only about half the programs and was valued more highly by trainees than program directors (p = 0.011). CONCLUSIONS: Training in procedural proficiency was viewed as successful by both program directors and graduates. Dialogue with, assessment by, and feedback from faculty were frequently used and most valued; stressing the importance of the facilitator role of faculty in the education of the trainee. These findings provide guidance for the development of newer methods of teaching and assessment.

Stories from early-career women physicians who have left academic medicine: a qualitative study at a single institution.

PURPOSE: The number of women in academic medicine has steadily increased, although gender parity still does not exist and women leave academics at somewhat higher rates than men. The authors investigated the reasons why women leave careers in academic medicine. METHOD: Semistructured, one-on-one interviews were conducted in 2007-2008 with 20 women physicians who had left a single academic institution to explore their reasons for opting out of academic careers. Data analysis was iterative, and an editing analysis style was used to derive themes. RESULTS: A lack of role models for combining career and family responsibilities, frustrations with research (funding difficulties, poor mentorship, competition), work-life balance, and the institutional environment (described as noncollaborative and biased in favor of male faculty) emerged as key factors associated with a decision to leave academic medicine for respondents. Faced with these challenges, respondents reevaluated their priorities and concluded that a discrepancy existed between their own and institutional priorities. Many respondents expressed divergent views with the institutional norms on how to measure success and, as a consequence, felt that they were undervalued at work. CONCLUSIONS: Participants report a disconnection between their own priorities and those of the dominant culture in academic medicine. Efforts to retain women faculty in academic medicine may include exploring the aspects of an academic career that they value most and providing support and recognition accordingly.

The culture of academic medicine: faculty perceptions of the lack of alignment between individual and institutional values.

BACKGROUND: Energized, talented faculty are essential to achieving the missions of academic medical centers (AMCs) in education, research and health care. The alignment of individuals' values with workplace experiences are linked to meaningfulness of work and productivity. OBJECTIVE: To determine faculty values and their alignment with institutional values. DESIGN: A qualitative hypothesis-generating interview study to understand the professional experiences of faculty and organizational approach in five AMCs that were nationally representative in regional and organizational characteristics. Analysis was inductive and data driven. PARTICIPANTS: Using stratified, purposeful sampling, we interviewed 96 male and female faculty at different career stages (early career, plateaued, senior faculty and those who had left academic medicine) and diverse specialties (generalists, medical and surgical subspecialists, and research scientists). APPROACH: Dominant themes that emerged from the data. RESULTS: Faculty described values relating to excellence in clinical care, community service (including care for the underserved and disadvantaged), teaching, intellectual rigor/freedom and discovery, all values that mirror the stated missions of AMCs. However, many faculty also described behaviors that led them to conclude that their AMCs, in practice, undervalued excellence in clinical care, and their social and educational missions. Themes were seen across gender, career stage, race and discipline, except that female leaders appeared more likely than male leaders to identify incongruence of individual values and organizational practices. CONCLUSIONS: In this study of five diverse medical schools, faculty values were well aligned with stated institutional missions; however, many perceived that institutional behaviors were not always aligned with individual faculty values.

Similarities and differences between international medical graduates and U.S. medical graduates at six Maryland community-based internal medicine residency training programs.

PURPOSE: Approximately 25% of practicing physicians in the United States are graduates of medical schools in other countries; they are called international medical graduates (IMGs). Their transition into the U.S. health care system may be difficult and challenging. This study sought to identify the similarities and differences between IMGs and U.S. medical graduates (USMGs) working together in residency training programs. METHOD: In 2006, the authors conducted a cross-sectional survey study of house officers (interns and residents) at six internal medicine (IM) community-based residency programs in Baltimore, Maryland. The survey asked about demographics, relocation for residency training, practice experience, and career plans and included four previously validated instruments: the Iowa Fatigue Scale, Cohen's Perceived Stress Scale, Rosenberg's Self-Esteem Scale, and the Personal Growth Scale. RESULTS: Of 225 potential house officers, 176 (78%) responded. In multivariable modeling, independent characteristics that differentiated IMGs from USMGs were that IMGs had (1) a native language other than English (odds ratio [OR] 18.3, 95% CI: 5.8-57.3), (2) less debt (<$50K) upon graduation from medical school (OR 7.3, 2.5-21.2), and (3) experiences practicing medicine before residency training (OR 41.02, 1.6-1017). With modeling to control for these three differences, the authors found IMGs to have lower fatigue (OR 2.7; 1.2-6.0), higher self-esteem (OR 3.0; 1.2-7.5), and greater personal growth scores (OR 3.6; 1.6-8.2). CONCLUSIONS: Differences exist between the IMGs and USMGs who are working together in community-based IM residency training programs. Considering such differences may help educators and program directors trying to support and train this diverse cadre of trainees.

Improving the diversity climate in academic medicine: faculty perceptions as a catalyst for institutional change.

PURPOSE: To assess perceptions of underrepresented minority (URM) and majority faculty physicians regarding an institution's diversity climate, and to identify potential improvement strategies. METHOD: The authors conducted a cross-sectional survey of tenure-track physicians at the Johns Hopkins University School of Medicine from June 1, 2004 to September 30, 2005; they measured faculty perceptions of bias in department/division operational activities, professional satisfaction, career networking, mentorship, and intentions to stay in academia, and they examined associations between race/ethnicity and faculty perceptions using multivariate logistic regression. RESULTS: Among 703 eligible faculty, 352 (50.1%) returned surveys. Fewer than one third of respondents reported experiences of bias in department/division activities; however, URM faculty were less likely than majority faculty to believe faculty recruitment is unbiased (21.1% versus 50.6%, P = .006). A minority of respondents were satisfied with institutional support for professional development. URM faculty were nearly four times less likely than majority faculty to report satisfaction with racial/ethnic diversity (12% versus 47.1%, P = .001) and three times less likely to believe networking included minorities (9.3% versus 32.6%, P = .014). There were no racial/ethnic differences in the quality of mentorship. More than 80% of respondents believed they would be in academic medicine in five years. However, URM faculty were less likely to report they would be at their current institution in five years (42.6% versus 70.5%, P = .004). CONCLUSIONS: Perceptions of the institution's diversity climate were poor for most physician faculty and were worse for URM faculty, highlighting the need for more transparent and diversity-sensitive recruitment, promotion, and networking policies/practices.

Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue.

BACKGROUND: Deficiencies in medical education research quality are widely acknowledged. Content, internal structure, and criterion validity evidence support the use of the Medical Education Research Study Quality Instrument (MERSQI) to measure education research quality, but predictive validity evidence has not been explored. OBJECTIVE: To describe the quality of manuscripts submitted to the 2008 Journal of General Internal Medicine (JGIM) medical education issue and determine whether MERSQI scores predict editorial decisions. DESIGN AND PARTICIPANTS: Cross-sectional study of original, quantitative research studies submitted for publication. MEASUREMENTS: Study quality measured by MERSQI scores (possible range 5-18). RESULTS: Of 131 submitted manuscripts, 100 met inclusion criteria. The mean (SD) total MERSQI score was 9.6 (2.6), range 5-15.5. Most studies used single-group cross-sectional (54%) or pre-post designs (32%), were conducted at one institution (78%), and reported satisfaction or opinion outcomes (56%). Few (36%) reported validity evidence for evaluation instruments. A one-point increase in MERSQI score was associated with editorial decisions to send manuscripts for peer review versus reject without review (OR 1.31, 95%CI 1.07-1.61, p = 0.009) and to invite revisions after review versus reject after review (OR 1.29, 95%CI 1.05-1.58, p = 0.02). MERSQI scores predicted final acceptance versus rejection (OR 1.32; 95% CI 1.10-1.58, p = 0.003). The mean total MERSQI score of accepted manuscripts was significantly higher than rejected manuscripts (10.7 [2.5] versus 9.0 [2.4], p = 0.003). CONCLUSIONS: MERSQI scores predicted editorial decisions and identified areas of methodological strengths and weaknesses in submitted manuscripts. Researchers, reviewers, and editors might use this instrument as a measure of methodological quality.

Long-term follow-up of a 10-month programme in curriculum development for medical educators: a cohort study.

CONTEXT: There is an ongoing need for curriculum development (CD) in medical education. However, only a minority of medical teaching institutions provide faculty development in CD. This study evaluates the long-term impact of a longitudinal programme in curriculum development. METHODS: We surveyed eight cohorts of participants (n = 64) and non-participants (n = 64) from 1988 to 1996 at baseline and at 6-13 years after completion of a 10-month, one half-day per week programme offered annually, which included a mentored CD project, workshops on CD steps, a final paper and a presentation. RESULTS: Fifty-eight participants (91%) and 50 non-participants (78%) returned completed follow-up surveys. In analyses, controlling for background characteristics and baseline self-rated proficiencies, participants were more likely than non-participants at follow-up to report having developed and implemented curricula in the past 5 years (65.5% versus 43.7%; odds ratio [OR] 2.41, 95% confidence interval [CI] 1.03-5.66), to report having performed needs assessment when planning a curriculum (86.1% versus 58.8%; OR 5.59, 95% CI 1.20-25.92), and to rate themselves highly in developing (OR 3.57, 95% CI 1.36-9.39), implementing (OR 3.04, 95% CI 1.16-7.93) and evaluating (OR 2.74, 95% CI 1.10-6.84) curricula. At follow-up, 86.2% of participants reported that the CD programme had made a moderate or great impact on their professional careers. Responses to an open-ended question on the impact confirmed continued involvement in CD work, confidence in CD skills, application of CD skills and knowledge beyond CD, improved time management, and lasting relationships formed because of the programme. CONCLUSIONS: Our results suggest that a longitudinal faculty development programme that engages and supports faculty in real CD work can have long-lasting impact.