Impact of a Regional Grant Program Through the Lens of Social Cognitive Career Theory: A Mixed-Method Evaluation.

PURPOSE: Evaluations of educational grant programs have focused on research productivity, with few examining impacts on grantees or effective program characteristics. This evaluation examined the regional grant program sponsored by Group on Educational Affairs to examine if and how grantees' careers were affected by funding, and if these experiences aligned with program goals. METHOD: In this concurrent, mixed-methods theory-driven evaluation, quantitative and qualitative data were analyzed independently and then integrated to examine complementarity. Quantitative data examined differences among 4 geographic regions and included proposal and grantee characteristics abstracted from administrative records of 52 funded proposals from 2010-2015 grant cycles. Qualitative data from 23 interviews conducted from 2018 to 2019 explored the impact on grantees, with Social Cognitive Career Theory (SCCT) serving as a framework for deductive thematic analysis. To facilitate integration of findings, quantitative data were layered onto each interview to permit exploration of associations between the 2 data types. RESULTS: Although significant regional differences existed in project length and amount of funding, there were few regional differences in grantee experiences. Despite small funding amounts, grants were perceived as career launching pads. The SCCT framework accounted for grantee experiences, including researcher identity formation and subsequent research, but did not capture collaboration phenomena. Integration of the 2 data types identified experience patterns unique to different groups of grantees (e.g., more or less research experience). The diversity among grantees suggests that clarification of program goals and stronger alignment with criteria for funding may be warranted. CONCLUSIONS: This evaluation illuminates why small educational grant programs may or may not impact interest and productivity in research. Implications exist for funders, including clarifying program goals and providing support for less experienced grantees. Future research should explore grantee subsets (e.g., underrepresented in medicine) to further identify what fosters or inhibits careers of medical education scholars.

Applying Conceptual and Theoretical Frameworks to Health Professions Education Research: An Introductory Workshop.

Introduction: Literature suggests that the quality and rigor of health professions education (HPE) research can be elevated if the research is anchored in existing theories and frameworks. This critical skill is difficult for novice researchers to master. We created a workshop to introduce the practical application of theories and frameworks to HPE research. Methods: We conducted two 60- to 75-minute workshops, one in 2019 at an in-person national conference and another in 2021 during an online national education conference. After a brief role-play introduction, participants applied a relevant theory to a case scenario in small groups, led by facilitators with expertise in HPE research. The workshop concluded with a presentation on applying the lessons learned when preparing a scholarly manuscript. We conducted a postworkshop survey to measure self-reported achievement of objectives. Results: Fifty-five individuals participated in the in-person workshop, and approximately 150 people completed the online workshop. Sixty participants (30%) completed the postworkshop survey across both workshops. As a result of participating in the workshop, 80% of participants (32) indicated they could distinguish between frameworks and theories, and 86% (32) could apply a conceptual or theoretical framework to a research question. Strengths of the workshop included the small-group activity, access to expert facilitators, and the materials provided. Discussion: The workshop has been well received by participants and fills a gap in the existing resources available to HPE researchers and mentors. It can be replicated in multiple settings to model the application of conceptual and theoretical frameworks to HPE research.

Research Methodologies in Health Professions Education Publications: Breadth and Rigor.

PURPOSE: Research methodologies represent assumptions about knowledge and ways of knowing. Diverse research methodologies and methodological standards for rigor are essential in shaping the collective set of knowledge in health professions education (HPE). Given this relationship between methodologies and knowledge, it is important to understand the breadth of research methodologies and their rigor in HPE research publications. However, there are limited studies examining these questions. This study synthesized current trends in methodologies and rigor in HPE papers to inform how evidence is gathered and collectively shapes knowledge in HPE. METHOD: This descriptive quantitative study used stepwise stratified cluster random sampling to analyze 90 papers from 15 HPE journals published in 2018 and 2019. Using a research design codebook, the authors conducted group coding processes for fidelity, response process validity, and rater agreement; an index quantifying methodological rigor was developed and applied for each paper. RESULTS: Over half of research methodologies were quantitative (51%), followed by qualitative (28%), and mixed methods (20%). No quantitative and mixed methods papers reported an epistemological approach. All qualitative papers that reported an epistemological approach (48%) used social constructivism. Most papers included participants from North America (49%) and Europe (20%). The majority of papers did not specify participant sampling strategies (56%) or a rationale for sample size (80%). Among those reported, most studies (81%) collected data within 1 year.The average rigor score of the papers was 56% (SD = 17). Rigor scores varied by journal categories and research methodologies. Rigor scores differed between general HPE journals and discipline-specific journals. Qualitative papers had significantly higher rigor scores than quantitative and mixed methods papers. CONCLUSIONS: This review of methodological breadth and rigor in HPE papers raises awareness in addressing methodological gaps and calls for future research on how the authors shape the nature of knowledge in HPE.

What Is an Innovation Article? A Systematic Overview of Innovation in Health Professions Education Journals.

PURPOSE: Innovation articles have their own submission category and guidelines in health professions education (HPE) journals, which suggests innovation might be a unique genre of scholarship. Yet, the requirements for innovation submissions vary among journals, suggesting ambiguity about the core content of this type of scholarship. To reduce this ambiguity, the researchers conducted a systematic overview to identify key features of innovation articles and evaluate their consistency in use across journals. Findings from this review may have implications for further development of innovation scholarship within HPE. METHOD: In this systematic overview, conducted in 2020, the researchers identified 13 HPE journals with innovation-type articles and used content analysis to identify key features from author guidelines and publications describing what editors look for in innovation articles. The researchers then audited a sample of 39 innovation articles (3/journal) published in 2019 to determine presence and consistency of 12 innovation features within and across HPE journals. Audit findings informed the researchers' evaluation of innovation as a genre in HPE. RESULTS: Findings show variability of innovation feature presence within and across journals. On average, articles included 7.8 of the 12 innovation features (SD 2.1, range 3-11). The most common features were description of: how the innovation was implemented (92%), a problem (90%), what was new or novel (79%), and data or outcomes (77%). On average, 5.5 (SD 1.5) out of 12 innovation features were consistently used in articles within each journal. CONCLUSIONS: The authors identified common features of innovation article types based on journal guidelines, but there was variability in presence and consistency of these features, suggesting HPE innovations are in an emerging state of genre development. The authors discuss potential reasons for variability within this article type and highlight the need for further discussion among authors, editors, and reviewers to improve clarity.

Challenges to assessing professional identity in medical students: a tale of two measures.

Background: Professional identity formation (PIF), a foundational process in becoming a physician, includes establishment of values, moral principles, and self-awareness. The purpose of this report is to examine challenges in establishing the validity of measures of identity fusion as one facet of PIF. Method: Utilizing the modern approach of validity as a unitary concept, the authors generated six hypotheses to examine the evidence for the construct validity of the scores of Physician Professional Identity (PPI) and Identity Integration (IdIn), considering relationships of these measures with each other, year of training and data from a larger survey. Results: Responses from 3473 students at 8 medical schools revealed a weak association between the measures with distributions varying by cohort. PPI had a stronger relationship to cohort and IdIn was moderately associated with students' attitudes relevant to social media use. Responses were independent of response format and evidence supported the interpretation of scores for IdIn as indications of integration of identity. Discussion : Sufficient evidence was found to suggest that these measures assess aspects of PIF. Use of these measures as part of a multidimensional, longitudinal approach to refining understanding of the construct of PIF and developing effective assessment strategies.

An Interprofessional Standardized Patient Case for Improving Collaboration, Shared Accountability, and Respect in Team-Based Family Discussions.

Introduction: The science of patient safety demonstrates that good communication is essential for effective interprofessional collaboration. Methods: We created a low-stakes, formative assessment with which medical students, pharmacy students, and nursing students could practice several of the Interprofessional Education Collaborative competencies. We aimed to enable students to practice collaborative care, respect for other disciplines, and shared accountability. Senior students from medicine, nursing, and pharmacy worked in teams to disclose a medical error to a standardized patient. The activity began with an icebreaker exercise wherein students learned about each other. Next, each team planned a strategy for error disclosure and collaboratively disclosed the error. Standardized patients evaluated the team's performance. Subsequently, students regrouped for a debriefing. The participating institutions administered a survey to their students. Results: In total, 1,151 students participated: 464 fourth-year students from the University of Houston College of Pharmacy, 450 third- and fourth-year students from Baylor College of Medicine, and 237 fourth-year students from Texas Woman's University Nelda C. Stark College of Nursing, all in Houston, Texas. Postsession survey data showed that students thought they achieved the relevant competencies. Students' understanding of the perspectives of the other two disciplines improved. Students found the simulation encounter and debriefing effective in helping them consider the contributions of other disciplines to patient care. Discussion: This interprofessional standardized patient activity enabled collaborative problem solving. The debriefing discussion broadened students' understanding of the expertise of the other disciplines and promoted shared accountability. Students found this activity engaging and effective.

A Review of Continuous Quality Improvement Processes at Ten Medical Schools.

The Liaison Committee on Medical Education now expects all allopathic medical schools to develop and adhere to a documentable continuous quality improvement (CQI) process. Medical schools must consider how to establish a defensible process that monitors compliance with accreditation standards between site visits. The purpose of this descriptive study is to detail how ten schools in the Association of American Medical Colleges' (AAMC) Southern Group on Educational Affairs (SGEA) CQI Special Interest Group (SIG) are tackling practical issues of CQI development including establishing a CQI office, designating faculty and staff, charging a CQI committee, choosing software for data management, if schools are choosing formalized CQI models, and other considerations. The information presented is not meant to certify that any way is the correct way to manage CQI, but simply present some schools' models. Future research should include defining commonalities of CQI models as well as seeking differences. Furthermore, what are components of CQI models that may affect accreditation compliance negatively? Are there "worst practices" to avoid? What LCME elements are most commonly identified for CQI, and what are the successes and struggles for addressing those elements? What are identifiable challenges relating to use of standard spreadsheet software and engaging information technology for support? How can students be more engaged and involved in the CQI process? Finally, how do these major shifts to a formalized CQI process impact the educational experience?

Using Jazz as a Metaphor to Teach Improvisational Communication Skills.

Metaphor helps humans understand complex concepts by "mapping" them onto accessible concepts. The purpose of this study was to investigate the effects of using jazz as a metaphor to teach senior medical students improvisational communication skills, and to understand student learning experiences. The authors designed a month-long course that used jazz to teach improvisational communication. A sample of fourth-year medical students (N = 30) completed the course between 2011 and 2014. Evaluation consisted of quantitative and qualitative data collected pre- and post-course, with comparison to a concurrent control group on some measures. Measures included: (a) Student self-reports of knowledge and ability performing communicative tasks; (b) blinded standardized patient assessment of students' adaptability and quality of listening; and (c) qualitative course evaluation data and open-ended interviews with course students. Compared to control students, course students demonstrated statistically significant and educationally meaningful gains in adaptability and listening behaviors. Students' course experiences suggested that the jazz components led to high engagement and creativity, and provided a model to guide application of improvisational concepts to their own communication behaviors. Metaphor proved to be a powerful tool in this study, partly through enabling increased reflection and decreased resistance to behaviors that, on the surface, tended to run counter to generally accepted norms. The use of jazz as a metaphor to teach improvisational communication warrants further refinement and investigation.

Near-peers improve patient safety training in the preclinical curriculum.

BACKGROUND: Accrediting bodies require medical schools to teach patient safety and residents to develop teaching skills in patient safety. We created a patient safety course in the preclinical curriculum and used continuous quality improvement to make changes over time. OBJECTIVE: To assess the impact of resident teaching on student perceptions of a Patient Safety course. DESIGN: Using the Institute for Healthcare Improvement patient safety curriculum as a frame, the course included the seven IHI modules, large group lectures and small group facilitated discussions. Applying a social action methodology, we evaluated the course for four years (Y1-Y4). RESULTS: In Y1, Y2, Y3 and Y4, we distributed a course evaluation to each student (n = 184, 189, 191, and 184, respectively) and the response rate was 96, 97, 95 and 100%, respectively. Overall course quality, clarity of course goals and value of small group discussions increased in Y2 after the introduction of residents as small group facilitators. The value of residents and the overall value of the course increased in Y3 after we provided residents with small group facilitation training. CONCLUSIONS: Preclinical students value the interaction with residents and may perceive the overall value of a course to be improved based on near-peer involvement. Residents gain valuable experience in small group facilitation and leadership.

Patient Safety Interprofessional Training for Medical, Nursing, and Pharmacy Students.

INTRODUCTION: Patient safety education is required in medical, nursing, and pharmacy training, and interprofessional education offers an ideal format for teaching the core concepts of patient safety. This training activity was developed to fulfill interprofessional education core competencies for communication and teamwork and was nested within a required patient safety course taught at a medical school. However, the activity can easily be adapted as a stand-alone offering that can be included in a preclinical doctoring course, offered as an elective, or hosted at a college of nursing or pharmacy. Our goal was to prepare learners for the clinical environment by providing a context for patient safety, communication, and teamwork. METHODS: Students participate in a 1.5-hour large-group activity that explores a case from the perspectives of each discipline. Faculty from all three disciplines sequentially present and debrief the case using focused questions to guide students' reflections and interactions between team members. RESULTS: We have presented this activity for 4 consecutive years. Students complete a questionnaire with retrospective pre-post ratings of their perspectives on the activity and its impact on their awareness of disciplinary roles and responsibilities, communication errors, and strategies for addressing interdisciplinary conflicts. Results show statistically significant increases in the items of interest. DISCUSSION: This interprofessional education offering is effective in terms of increasing awareness and knowledge among members of three health care disciplines, improving awareness of potential kinds of communication errors, and helping students consider the role of interdisciplinary interactions.

Meaningful Learning Moments on a Family Medicine Clerkship: When Students Are Patient Centered.

BACKGROUND AND OBJECTIVES: Reflection after patient encounters is an important aspect of clinical learning. After our medical school instituted a reflection paper assignment for all clerkships, we wanted to learn about the types of encounters that students found meaningful on a family medicine clerkship and how they impacted students' learning. METHODS: Family and Community Medicine Clerkship students completed a reflection paper after the clerkship, based on guidelines that were used for all clerkship reflection papers at our medical school. Two reviewers independently organized student responses into themes and then jointly prioritized common themes and negotiated any initial differences into other themes. RESULTS: A total of 272 reflection papers describing an actual learning moment in patient care were submitted during the study period of January 2011--December 2012. In describing actions performed, students most frequently wrote about aspects of patient-centered care such as listening to the patient, carefully assessing the patient's condition, or giving a detailed explanation to the patient. In describing effects of those actions, students wrote about what they learned about the patient-physician interaction, the trust that patients demonstrated in them, the approval they gained from their preceptors, and the benefits they saw from their actions. CONCLUSIONS: An important contribution of a family medicine clerkship is the opportunity for students to further their skills in patient-centered care and realize the outcomes of providing that type of care.

A guiding framework to maximise the power of the arts in medical education: a systematic review and metasynthesis.

CONTEXT: A rich literature describes many innovative uses of the arts in professional education. However, arts-based teaching tends to be idiosyncratic, depending on the interests and enthusiasm of individual teachers, rather than on strategic design decisions. An overarching framework is needed to guide implementation of arts-based teaching in medical education. The objective of this study was to review and synthesise the literature on arts-based education and provide a conceptual model to guide design, evaluation and research of the use of the arts in medical education. METHODS: A systematic literature review using the PubMed and ERIC databases. Search terms included humanism, art, music, literature, teaching, education, learning processes, pedagogy and curriculum. We selected empirical studies and conceptual articles about the use of creative arts, imagery and symbolism in the context of professional education. Data synthesis involved a qualitative content analysis of 49 included articles, identifying themes related to educational characteristics, processes and outcomes in arts-based education. RESULTS: Four common themes were identified describing (i) unique qualities of the arts that promote learning, (ii) particular ways learners engage with art, (iii) documented short- and long-term learning outcomes arising from arts-based teaching and (iv) specific pedagogical considerations for using the arts to teach in professional education contexts. CONCLUSIONS: The arts have unique qualities that can help create novel ways to engage learners. These novel ways of engagement can foster learners' ability to discover and create new meanings about a variety of topics, which in turn can lead to better medical practice. At each of these steps, specific actions by the teacher can enhance the potential for learners to move to the next step. The process can be enhanced when learners participate in the context of a group, and the group itself can undergo transformative change. Future work should focus on using this model to guide process design and outcome measurement in arts-based education.

Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review.

PURPOSE: Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies. METHOD: The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality. RESULTS: Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills. CONCLUSIONS: These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development.

The utility of reflective writing after a palliative care experience: can we assess medical students' professionalism?

BACKGROUND: Medical education leaders have called for a curriculum that proactively teaches knowledge, skills, and attitudes required for professional practice and have identified professionalism as a competency domain for medical students. Exposure to palliative care (PC), an often deeply moving clinical experience, is an optimal trigger for rich student reflection, and students' reflective writings can be explored for professional attitudes. OBJECTIVE: Our aim was to evaluate the merit of using student reflective writing about a PC clinical experience to teach and assess professionalism. METHODS: After a PC patient visit, students wrote a brief reflective essay. We explored qualitatively if/how evidence of students' professionalism was reflected in their writing. Five essays were randomly chosen to develop a preliminary thematic structure, which then guided analysis of 30 additional, randomly chosen essays. Analysts coded transcripts independently, then collaboratively, developed thematic categories, and selected illustrative quotes for each theme and subtheme. RESULTS: Essays revealed content reflecting more rich information about students' progress toward achieving two professionalism competencies (demonstrating awareness of one's own perspectives and biases; demonstrating caring, compassion, empathy, and respect) than two others (displaying self-awareness of performance; recognizing and taking actions to correct deficiencies in one's own behavior, knowledge, and skill). CONCLUSIONS: Professional attitudes were evident in all essays. The essays had limited use for formal summative assessment of professionalism competencies. However, given the increasing presence of PC clinical experiences at medical schools nationwide, we believe this assessment strategy for professionalism has merit and deserves further investigation.

Fostering students' reflection about bias in healthcare: cognitive dissonance and the role of personal and normative standards.

BACKGROUND: To reduce cognitive dissonance about one's beliefs or behavior, individuals may compare their behavior to personal and/or normative standards. The details of this reflection process are unclear. AIMS: We examined how medical students compare their behavior or beliefs to standards in discussions about implicit bias, and explored if and how different reflective pathways (preserving vs. reconciling) are associated with each standard. METHODS: Third-year students engaged in a small-group discussion about bias. Some students and group facilitators also participated in a debriefing about the experience. Using qualitative methods, the transcripts from these 11 sessions were analyzed for evidence of student comparison to a standard and of reflection pathways. RESULTS: Of 557 text units, 75.8% could be coded with a standard and/or a path of reflection. Students referenced personal and normative standards about equally, and preserved or reconciled existing beliefs about equally. Uses of normative standards were associated with preservation-type reflection, and uses of personal standards with reconciliation-type reflection. CONCLUSIONS: Normative expectations of physicians are sometimes used to provoke students' consideration of implicit biases about patients. To encourage critical reflection and reconciliation of biased beliefs or behavior, educators should frame reflective activities as a personal exercise rather than as a requirement.

Measuring the quality of patients' goals and action plans: development and validation of a novel tool.

BACKGROUND: The purpose of this study is to develop and test reliability, validity, and utility of the Goal-Setting Evaluation Tool for Diabetes (GET-D). The effectiveness of diabetes self-management is predicated on goal-setting and action planning strategies. Evaluation of self-management interventions is hampered by the absence of tools to assess quality of goals and action plans. To address this gap, we developed the GET-D, a criteria-based, observer rating scale that measures the quality of patients' diabetes goals and action plans. METHODS: We conducted 3-stage development of GET-D, including identification of criteria for observer ratings of goals and action plans, rater training and pilot testing; and then performed psychometric testing of the GET-D. RESULTS: Trained raters could effectively rate the quality of patient-generated goals and action plans using the GET-D. Ratings performed by trained evaluators demonstrated good raw agreement (94.4%) and inter-rater reliability (Kappa = 0.66). Scores on the GET-D correlated well with measures theoretically associated with goal-setting, including patient activation (r=.252, P<.05), diabetes specific self-efficacy (r=.376, P<.001) and inverse relationship with depression (r= -.376, P<.01). Significant between group differences (P<.01) in GET-D scores between goal-setting intervention (mean = 7.33, standard deviation = 4.4) and education groups (mean = 4.93, standard deviation = 3.9) confirmed construct validity of the GET-D. CONCLUSIONS: The GET-D can reliably and validly rate the quality of goals and action plans. It holds promise as a measure of intervention fidelity for clinical interventions that promote diabetes self-management behaviors to improve clinical outcomes. TRIAL REGISTRATION: Clinicaltrials.gov Identifier: NCT00481286.

A standards-based, peer-reviewed teaching award to enhance a medical school's teaching environment and inform the promotions process.

The authors provide the rationale, design, and description of a unique teaching award that has enhanced Baylor College of Medicine's teaching environment and become highly valued by the promotions and tenure (P&T) committee in determining a faculty member's readiness for promotion. This award is self-nominating and standards based. The primary purpose for development of the award was to provide the Baylor community and the P&T committee a method to understand and value the scholarship of teaching to the same degree that they understand and value the scholarship of discovery.The authors also present results from an internal evaluation of the program that included a survey and interviews. Between the inception of the award in 2001 and the internal review conducted in 2010, the award could have had an influence on the promotion of 130 of the recipients. Of the 130, 88 (65.6%) received this award before gaining their current rank (χ (1) = 16.3, P < .001). Stakeholders, including department chairs and members of the P&T committee, agreed that this award is valuable to those seeking promotion. Individual recipients stated that the award is good for the institution by encouraging reflection on teaching; increasing the recognition, importance, and value of teaching; encouraging the improvement of teaching skills; and providing a better understanding to others about what medical teachers really do. Of the 214 open-ended responses to survey questions of award recipients, more than half the comments were about the value of the award and its positive effect on promotion.