Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments.

Introduction: There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how to develop new instruments that will yield valid data. Methods: The literature, consensus findings of curricula and content experts, and principles of adult learning guided the content and methodology of the workshop. The workshop underwent stringent peer review prior to presentation at one international and three national academic conferences. In the interactive workshop, selected domains of validity evidence were taught with sequential cycles of didactics, demonstration, and deliberate practice with facilitated feedback. An exercise guide steered participants through a stepwise approach. Using Likert-scale items and open-response questions, an evaluation form rated the workshop's effectiveness, captured details of how learners reached the objectives, and determined participants' plans for future work. Results: The workshop demonstrated generalizability with successful implementation in diverse settings. Sixty-five learners, the majority being clinician-educators, completed evaluations. Learners rated the workshop favorably for each prompt. Qualitative comments corroborated the workshop's effectiveness. The active application and facilitated feedback components allowed learners to reflect in real time as to how they were meeting a particular objective. Discussion: This feasible and practical educational intervention fills a literature gap by showing the medical educator how to apply validity evidence to both existing and in-development assessment instruments. Thus, it holds the potential to significantly impact learner and, subsequently, patient outcomes.

Medical Student Participation in Patient- and Family-Centered Rounding: A National Survey of Pediatric Clerkships.

OBJECTIVE: Pediatrics rotations may be medical students' only experience with patient- and family-centered rounding (PFCR). It is unclear how students participate in or are prepared for PFCR. We surveyed national pediatrics clerkships to determine the prevalence of PFCR and the proportion providing orientation in order to inform a needs assessment for PFCR orientation. METHODS: A 5-item peer-reviewed survey was distributed to the Council on Medical Student Education in Pediatrics (COMSEP) membership as part of a larger survey in 2017. Institutional differences among programs performing PFCR were compared using chi-square and t-tests. Responses to 1 open-ended question were coded and grouped into broad categories using content analysis. RESULTS: The full COMSEP survey received answers from 190 participants representing 103 medical schools. Our questions received 174 responses representing 94 schools (98 training sites) and had an 85% (83/98) prevalence of student PFCR participation. Although most (n = 108; 85%) reported that their students received PFCR orientation, half (n = 62; 49%) considered orientation "informal," and only 2 reported using published curricula. After didactics, the most common orientation materials were handouts (n = 33; 26%), videos (n = 13; 10%), and role play (n = 7; 6%). Orientation was most commonly initiated at the start of clerkship (n = 62; 49%) by clerkship administration (n = 38; 30%), but 20% (n = 26) reported resident-led orientation. Qualitative responses (n = 98) were coded and organized into 4 themes; the greatest perceived challenges for medical students on PFCR were communication and anxiety. CONCLUSIONS: Although most students participate in and receive orientation to PFCR, there is wide variability in the content, timing, and administration of orientation. A nationally disseminated, evidence-based orientation curriculum may reduce educational variability and better prepare students for PFCR.

Innovation to Dissemination Workshop: Selecting Outcome Measures to Translate Educational Innovations Into Scholarship.

Introduction: Curricular innovations are invaluable to the improvement of medical education programs, and thus, their dissemination to broader audiences is imperative. However, medical educators often struggle to translate innovative ideas into scholarly pursuits due to a lack of experience or expertise in selecting outcome measures that demonstrate impact. A recent national call for increased focus on outcome measures for medical education research highlights the need for more training in this area. Methods: We developed a 2-hour interactive workshop to improve educator ability to identify outcome measures for educational innovations. This workshop was delivered at a national pediatrics educational conference and at three local institutional faculty development sessions. Results: Participants were diverse in terms of experience, expertise, and roles within their educational programs. Participants rated the workshop positively in each setting and identified next steps in developing their own products of educational scholarship. Discussion: This workshop can provide faculty and faculty developers with a template for developing a skill set in identifying outcome measures and pairing them with educational innovations.