What Do Pediatric Subinterns Say About Their Learning and Assessment? A Qualitative Analysis of Individual Learning Plans.

OBJECTIVE: To perform a qualitative content analysis of learning and assessment strategies that pediatric subinterns describe in Individualized Learning Plans (ILPs) and to explore barriers and facilitators to their learning. METHODS: We analyzed ILPs from medical students enrolled in pediatric subinternships at 10 US medical schools that utilized a standardized curriculum and were recruited to reflect diversity in geographic location, funding, and enrollment. Students used an ILP to record 3 or more selected learning objectives, rationale for selection, and reflection on learning and assessment strategies. Investigators used the constant comparative method to perform a content analysis of the ILPs, grouping codes into themes, and verifying relationships between codes within themes. RESULTS: Two hundred and four ILPs that included student reflections on 850 learning objectives were analyzed. Content was analyzed in 5 categories: rationale for selecting objectives, learning strategies, assessment strategies, challenges to learning, and facilitators of learning. Students showed strong commitment to individualized, self-directed learning, developed a wide range of creative learning strategies, and relied heavily on self-reflection to assess their progress. The learning environment both helped and hindered students' ability to make and assess progress on their selected learning objectives. CONCLUSIONS: Through ILP-guided reflection and a formal curriculum, students can choose well-justified learning objectives and demonstrate resourcefulness and independence in developing self-directed learning and assessment strategies. The strategies that students identified in this study provide a menu of learning and assessment options for subinterns. Identified challenges and facilitators of learning provide guidance for educators who seek to enhance the clinical learning environment.

Who Wants to Learn How to Teach? Perceptions of Radiology Residents and Radiology Teaching Faculty Regarding Resident as Teacher Training.

RATIONALE AND OBJECTIVES: While the ACGME requires Resident as Teacher (RAT) training, curricula in radiology remain limited. Our study was performed to examine radiology residents (RR) and teaching faculty (TF) perceptions about RAT training. MATERIALS AND METHODS: In 2021, anonymous online surveys were administered to all RR (53-item) and to all TF (24-item) of a radiology residency program. Content domains included attitudes about RAT training and learning topics. RESULTS: Response rates were 97% (38/39) for RR and 54% (58/107) for TF. Most RR desired training to become better educators to medical students (MS) (81%) and other residents (83%). Seventy-seven percent of RR reported the importance regarding how to give feedback to other learners, while 94% desired formal training on delivering case presentations. While 94% of RR reported that resident feedback was valuable, only 6% reported always giving feedback to MS. Seventy-two percent of RR did not apply at least some best-practices in their reading room teaching. Fifty-nine percent of RR wanted TF to observe their own teaching skills and provide feedback although 70% reported rarely or never receiving TF feedback. Ninety-three percent of TF reported RR should receive RAT training, while 88% reported that feedback of RR to MS was important. CONCLUSION: RR and TF strongly endorsed the need for RAT training. RR anticipate teaching to be an important part of their careers. We identified learning topics and possible gaps regarding how TF are meeting RR needs, which could inform the development of RAT curricula.

Can We Ensure That Workshops Are Effective in Their Goal? Impact of a National Education Workshop on Participants' Subsequent Scholarship.

Medical education conferences offer practical workshops to facilitate physicians' lifelong learning. Little is known about integration of workshop material after conferences. We sought to evaluate the application of workshop content focused on scholarly publication preparation. We developed an email survey to examine participants' progress preparing a publication in 2019, administered 4, 9, and 15 months post-conference. The survey included scaled items and open-ended questions. Thirty-three participants attended the workshop. Participants continued to develop their projects, but noted time, adequate evaluations, and no writing partners as barriers. Following up with workshop participants offers insights into effective application of workshop content.

Teaching Sexual Orientation and Gender Identity in Pediatric Clinical Settings: A Training Workshop for Faculty and Residents.

Introduction: Health disparities for the lesbian, gay, bisexual, transgender, queer, intersex, asexual, all other genders, sexes, and sexualities (LGBTQIA+) population are striking. Yet, deliberate efforts to integrate sexual orientation and gender identity in pediatric education settings remain lacking. The type of formal training that pediatric educators currently have for teaching of sexual orientation and gender identity is unclear and limited, which led to the development and implementation of this curriculum. Methods: A 2-hour workshop was developed to address gaps in knowledge, equip faculty and resident educators with skills to apply key concepts in teaching activities, and motivate them to examine challenges and opportunities in teaching sexual orientation and gender identity principles in their routine duties in pediatric settings across the undergraduate and graduate education spectrum. Learning strategies of the workshop included learner activation, a didactic, and clinical cases with role-play opportunities. Participants completed evaluations at the end of the workshop. Results: The workshop was implemented in three varied educational settings in 2019. All 65 participants enrolled in the workshop completed the evaluations. Evaluations ranged from 4.6 to 4.9 on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Participants reported workshop strengths and anticipated impact on their own teaching and clinical practice. Discussion: Stark health disparities for the LGBTQIA+ population and gaps in relevant curricula demand a training intervention for pediatric educators. We demonstrated the successful implementation of a training workshop, with evidence of feasibility and generalizability, that addressed knowledge gaps and teaching and clinical skills.

The Development, Implementation, and Evaluation of an Acute Otitis Media Education Website.

OBJECTIVE: To develop, implement, and evaluate an acute otitis media (AOM) education website for clinician-educators. METHODS: We developed an education website following Kern's curriculum model. RESULTS: The website contained peer-reviewed content, educational objectives, library search pages to identify evidence-based resources, and a faculty toolbox with instructional and evaluation instruments. Pediatric clinician-educators were purposefully sampled from different clinic sites to evaluate the website. Semistructured interviews explored key website components for content and usability in clinical teaching. In grounded theory tradition, investigators used the constant comparative method with qualitative analysis software to identify themes and representative quotations. Eleven faculty members (9 females and 2 males with teaching experience from 6 to 26 years) participated in the study. Identified themes were: 1) value of visual impact for learning specific topics, 2) promotion of efficiency in teaching clinical topics, 3) varying approaches for using website, and 4) faculty's self-report of knowledge and self-efficacy needs. CONCLUSIONS: An education website may enhance the teaching of AOM, accommodate different teaching preferences, promote efficiency in teaching, and advance clinician-educator knowledge and skills. Next steps include evaluation of learners' perspectives, generalizability in varied teaching settings, and assessment of higher learning outcomes including impact on knowledge, skills, and patient outcomes.

Teaching pediatric otoscopy skills to the medical student in the clinical setting: preceptor perspectives and practice.

BACKGROUND: Acute otitis media (AOM) is the most frequent indication for antibiotic treatment of children in the United States. Its diagnosis relies on visualization of the tympanic membrane, a clinical skill acquired through a deliberate approach. Instruction in pediatric otoscopy begins in medical school. Medical students receive their primary experience with pediatric otoscopy during the required pediatric clerkship, traditionally relying on an immersion, apprentice-type learning model. A better understanding of their preceptors' clinical and teaching practices could lead to improved skill acquisition. This study investigates how pediatric preceptors (PP) and members of the Council on Medical Student Education in Pediatrics (COMSEP) perceive teaching otoscopy. METHODS: A 30-item online survey was administered to a purposeful sample of PP at six institutions in 2017. A comparable 23-item survey was administered to members through the 2018 COMSEP Annual Survey. Only COMSEP members who identified themselves as teaching otoscopy to medical students were asked to complete the otoscopy-related questions on the survey. RESULTS: Survey respondents included 58% of PP (180/310) and 44% (152/348) of COMSEP members. Forty-one percent (62/152) of COMSEP member respondents identified themselves as teaching otoscopy and completed the otoscopy-related questions. The majority agreed that standardized curricula are needed (PP 78%, COMSEP members 97%) and that all graduating medical students should be able to perform pediatric otoscopy (PP 95%, COMSEP members 79%). Most respondents reported usefulness of the American Academy of Pediatrics (AAP) AOM guidelines (PP 95%, COMSEP members 100%). More COMSEP members than PP adhered to the AAP's diagnostic criteria (pediatric preceptors 42%, COMSEP members 93%). The most common barriers to teaching otoscopy were a lack of assistive technology (PP 77%, COMSEP members 56%), presence of cerumen (PP 58%, COMSEP members 60%), time to teach in direct patient care (PP 46%, COMSEP members 48%), and parent anxiety (PP 62%, COMSEP members 54%). CONCLUSIONS: Our study identified systemic and individual practice patterns and barriers to teaching pediatric otoscopy. These results can inform education leaders in supporting and enabling preceptors in their clinical teaching. This approach can be adapted to ensure graduating medical students obtain intended core clinical skills.

The Decline in Community Preceptor Teaching Activity: Exploring the Perspectives of Pediatricians Who No Longer Teach Medical Students.

PURPOSE: Difficulty in recruiting and retaining community preceptors for medical student education has been described in the literature. Yet little, if any, information is known about community outpatient preceptors who have stopped or decreased teaching time with students. This study aimed to examine these preceptors' perspectives about this phenomenon. METHOD: Using a phenomenology framework, this multi-institutional qualitative study used semistructured interviews with community pediatric preceptors who had stopped or reduced teaching time with medical students. Interviews were conducted between October 2017 and January 2018 and transcribed verbatim. Interviews explored factors for engaging in teaching, or decreasing or ceasing teaching, that would enable future teaching. An initial code book was developed and refined as data were analyzed to generate themes. RESULTS: Twenty-seven community pediatricians affiliated with 10 institutions participated. Thirty-seven codes resulted in 4 organizing themes: evolution of health care, personal barriers, educational system, and ideal situations to recruit and retain preceptors, each with subthemes. CONCLUSIONS: From the viewpoints of physicians who had decreased or stopped teaching students, this study more deeply explores previously described reasons contributing to the decline of community preceptors, adds newly described barriers, and offers strategies to help counter this phenomenon based on preceptors' perceptions. These findings appear to be manifestations of deeper issues including the professional identify of clinical educators. Understanding the barriers and strategies and how they relate to preceptors themselves should better inform education leaders to more effectively halt the decline of community precepting and enhance the clinical precepting environment for medical students.

Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments.

Introduction: There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how to develop new instruments that will yield valid data. Methods: The literature, consensus findings of curricula and content experts, and principles of adult learning guided the content and methodology of the workshop. The workshop underwent stringent peer review prior to presentation at one international and three national academic conferences. In the interactive workshop, selected domains of validity evidence were taught with sequential cycles of didactics, demonstration, and deliberate practice with facilitated feedback. An exercise guide steered participants through a stepwise approach. Using Likert-scale items and open-response questions, an evaluation form rated the workshop's effectiveness, captured details of how learners reached the objectives, and determined participants' plans for future work. Results: The workshop demonstrated generalizability with successful implementation in diverse settings. Sixty-five learners, the majority being clinician-educators, completed evaluations. Learners rated the workshop favorably for each prompt. Qualitative comments corroborated the workshop's effectiveness. The active application and facilitated feedback components allowed learners to reflect in real time as to how they were meeting a particular objective. Discussion: This feasible and practical educational intervention fills a literature gap by showing the medical educator how to apply validity evidence to both existing and in-development assessment instruments. Thus, it holds the potential to significantly impact learner and, subsequently, patient outcomes.

Teaching Pediatric Otoscopy Skills to Pediatric and Emergency Medicine Residents: A Cross-Institutional Study.

OBJECTIVE: To evaluate a pediatric otoscopy curriculum with the use of outcome measures that included assessment of skills with real patients. METHODS: Thirty-three residents in an intervention group from 2 institutions received the curriculum. In the previous year, 21 residents in a nonintervention group did not receive the curriculum. Both groups were evaluated at the beginning and end of their internship years with the use of the same outcome assessments: 1) a written test, 2) an objective standardized clinical examination (OSCE), and 3) direct observation of skills in real patients with the use of a checklist with established validity. RESULTS: The intervention group had a significant increase in percentage reaching minimum passing levels between the beginning and end of the internship year for the written test (12% vs 97%; P < .001), OSCE (0% vs 78%; P < .001), and direct observation (0% vs 75%; P < .001); significant mean percentage gains for the written test (21%; P < .001), OSCE (28%; P < .001), and direct observation (52%; P = .008); and significantly higher (P < .001) mean percentage gains than the nonintervention group on the written test, OSCE, and direct observation. The nonintervention group did not have a significant increase (P = .99) in percentage reaching minimum passing levels, no significant mean percentage gains in the written test (2.7%; P = .30) and direct observation (6.7%; P = .61), and significant regression in OSCE (-5.2%; P = .03). CONCLUSIONS: A pediatric otoscopy curriculum with multimodal outcome assessments was successfully implemented across different specialties at multiple institutions and found to yield gains, including in skills with real patients.

Conducting Quantitative Medical Education Research: From Design to Dissemination.

Rigorous medical education research is critical to effectively develop and evaluate the training we provide our learners. Yet many clinical medical educators lack the training and skills needed to conduct high-quality medical education research. We offer guidance on conducting sound quantitative medical education research. Our aim is to equip readers with the key skills and strategies necessary to conduct successful research projects, highlighting new concepts and controversies in the field. We utilize Glassick's criteria for scholarship as a framework to discuss strategies to ensure that the research question of interest is worthy of further study and how to use existing literature and conceptual frameworks to strengthen a research study. Through discussions of the strengths and limitations of commonly used study designs, we expose the reader to particular nuances of these decisions in medical education research and discuss outcomes generally focused on, as well as strategies for determining the significance of consequent findings. We conclude with information on critiquing research findings and preparing results for dissemination to a broad audience. Practical planning worksheets and comprehensive tables illustrating key concepts are provided in order to guide researchers through each step of the process. Medical education research provides wonderful opportunities to improve how we teach our learners, to satisfy our own intellectual curiosity, and ultimately to enhance the care provided to patients.

Teaching the pediatric ear exam and diagnosis of Acute Otitis Media: a teaching and assessment model in three groups.

BACKGROUND: The serious consequences of inaccurate diagnosis of acute otitis media have led to a call for greater education to develop proficient pediatric otoscopy skills. Despite the clinical and educational needs, peer-reviewed standardized curricula with validated assessment instruments remain limited. This study evaluated a pediatric otoscopy curriculum incorporated into the Pediatric medical student clerkship with use of outcome measures that included assessment of skills with real patients. The objective was to determine whether students who received the intervention would demonstrate significant gains in pediatric otoscopy skills when compared with students with only routine immersion learning exposure. METHODS: During their Pediatric clerkship, an intervention group (IG) of 100 medical students received routine instruction and a curriculum intervention. A non-intervention group (NIG) of 30 students received only routine instruction. Outcome measures included written tests and assessment of skills with real patients. A retention group (RG) consisted of 79 students in the IG who completed a written test at the end of medical school. Paired t-tests were used to compare differences in pre-intervention, post-intervention, and retention scores for the IG, NIG, and RG, while analysis of covariance tests were used to compare differences in scores between the IG and NIG. RESULTS: Pre-intervention scores were similar for the IG and NIG for the written test (mean/SD of 12.9/2.9 for IG and 12.9/1.8 for NIG, p = 0.78) and skills checklist (mean/SD of 11.1/4.4 for IG and 10.9/4.0 for NIG, p = 0.88). The IG had significantly higher post-intervention scores than the NIG for the written test (mean/SD of 22.6/1.7 for IG and 13.9/2.5 for NIG, p < 0.001) and skills checklist (mean/SD of 19.2/3.4 for IG and 11.0/3.8 for NIG, p < 0.001). The IG also had significantly higher gain in scores than the NIG for the written test (mean/SD +9.6/2.8 for IG and +1.0/2.3 for NIG, p < 0.001) and skills checklist (mean/SD of +8.1/4.8 for IG and +0.1/4.5 for NIG, p < 0.001). For the RG, there was a significant decrease (p < 0.001) from the post-intervention scores to retention scores (mean/SD of -7.4/2.7) but a significant increase (p < 0.001) from the pre-intervention score to retention score (mean + 2.6/3.3). CONCLUSIONS: Medical students who received a formal curriculum intervention demonstrated significant gains in pediatric otoscopy skills when compared with students with only routine immersion learning exposure. However, learning gains diminished over time, emphasizing the need for continued practice opportunities to reinforce students' skills. Our study provides a formal curriculum to meet identified educational gaps in the important topic of pediatric otoscopy and offers a model for teaching of other clinical skills using rigorous outcome measures including assessment of skills in real patients.

Recruiting and Retaining Community-Based Preceptors: A Multicenter Qualitative Action Study of Pediatric Preceptors.

PURPOSE: The recruitment and retention of community preceptors to teach medical students is difficult. The authors sought to characterize the underlying motivational factors for becoming a preceptor and to identify strategies for recruiting and retaining community-based pediatric preceptors. METHOD: This multicenter qualitative action study included semistructured interviews with community-based pediatric preceptors affiliated with 12 institutions from August to December 2015. Only active preceptors were included, and participating institutions were diverse with respect to geographic location and class size. Interviews were conducted over the telephone and transcribed verbatim. Six investigators used deidentified transcripts to develop a codebook. Through a constant comparative method, codes were revised as data were analyzed and disagreements were resolved through discussion. All investigators organized the themes into dimensions. RESULTS: Fifty-one preceptors were interviewed. Forty-one themes coalesced into four dimensions: (1) least liked aspects of teaching, (2) preparation to teach, (3) inspiration to teach, and (4) ways to improve recruitment and retention. Time constraints and patient care demands were the most commonly cited deterrents to teaching. Successful preceptors balanced their clinical demands with their desire to teach using creative scheduling. External rewards (e.g., recognition, continuing medical education credit) served as incentives. Internal motivation inspired participants to share their enthusiasm for pediatrics and to develop longitudinal relationships with their learners. CONCLUSIONS: Changes in health care delivery have imposed more time constraints on community-based preceptors. However, this study identified underlying factors motivating physicians to volunteer as preceptors. Strategies to recruit new and retain current preceptors must be collaborative.

Standardized Checklist for Otoscopy Performance Evaluation: A Validation Study of a Tool to Assess Pediatric Otoscopy Skills.

INTRODUCTION: Acute otitis media (AOM) is the most frequently diagnosed pediatric illness in the United States and the most common indication for antibiotic prescription. Skill in pediatric otoscopy is essential to correctly identify children with AOM. However, studies have found diagnostic inconsistency and significant overdiagnosis among practitioners. Given the potential public and individual health consequences, there has been a call for improved education regarding the diagnostic certainty of AOM. Yet educational efforts continue to be limited, particularly in regard to competency assessment. The lack of a validated tool to assess otoscopy skill attainment objectively diminishes the instructor's ability to provide useful feedback and direction to the learner. METHODS: We have undertaken an educational intervention with the goal of developing a validated Standardized Checklist for Otoscopy Performance Evaluation (SCOPE), building on key principles of the general pediatric ear exam. The SCOPE was developed with the input of process and content experts with attention to specific domains of validity. RESULTS: Our analysis provides important validity evidence for the SCOPE assessment tool. The instrument was piloted and successfully implemented with medical students and varying levels of residents in pediatrics and emergency medicine over a 5-year period in varied settings: urgent care, large and small pediatric clinics, and the emergency departments at two institutions. It has been used for both instruction and assessment purposes. DISCUSSION: Because the SCOPE can be used in teaching demonstration purposes, in formative and summative assessment settings, and across the continuum of learners, this instrument offers the potential for more educational efforts in the field of assessment in direct patient care. We anticipate that the SCOPE will foster an environment of efficient yet high-yield review and discussion of otoscopy and diagnostic competency.

Evaluation of a culturally effective health care curriculum integrated into a core pediatric clerkship.

OBJECTIVE: The aim of this study was to evaluate a culturally effective health care (CEHC) curriculum integrated into the real-time clinical experience of a third-year medical student pediatric clerkship. METHODS: The intervention group (n = 22) and the nonintervention group (n = 69) consisted of students who were assigned to one of two sites for their clerkship. Students did not volunteer for the curriculum. A curriculum in 2002 was developed based upon a local needs assessment of students and parents, key CEHC concepts and experts' input. Learning strategies included incorporation of CEHC issues into clinic precepting, attending rounds, and written histories. Evaluation methods were preintervention and postintervention knowledge tests and Likert-type attitudinal surveys, and a final objective structured clinical exam (OSCE; nonintervention group, n = 22, intervention group, n = 22). RESULTS: Pretest knowledge scores were similar in both groups. The post-test scores were significantly different. The intervention group demonstrated higher gain in the knowledge scores (42% vs 5%; P < .001). The intervention group demonstrated significantly higher gains in observed role modeling (85% vs 31%; P = .01), self-perceived skill (82% vs 19%; P < .001), and attitude (21% vs 0%; P = .02), but not in self-perceived knowledge domains (65% vs 15%; P = .14) on the attitudinal survey. The intervention group performed significantly better in the folk (83% vs 70%; P = .02) and language (75% vs 63%; P = .01) OSCE stations and had a significantly higher total OSCE score (79% vs 68%; P = .01). CONCLUSION: A CEHC curriculum, stressing clinical relevance, was successfully incorporated into the real-time experience of a third-year medical student pediatric clerkship. Students demonstrated significant gains in knowledge, attitudinal domains, and clinical skills.