Learner Levels of Supervision Across the Continuum of Pediatrics Training.

PURPOSE: To describe trajectories in level of supervision ratings for linked entrustable professional activities (EPAs) among pediatric learners in medical school, residency, fellowship. METHOD: The authors performed secondary analyses of 3 linked datasets of level of supervision ratings for the Core EPAs for Entering Residency, the General Pediatrics EPAs, and the Subspecialty Pediatrics EPAs. After identifying 9 activities in common across training stages and aligning the level of entrustment-supervision scales across the datasets, piecewise ordinal and linear mixed effects models were fitted to characterize trajectories of supervision ratings. RESULTS: Within each training period, learners were rated as needing less supervision over time in each activity. When transitioning from medical school to residency or during the first year of residency, learners were rated as needing greater supervision in activities related to patient management, teamwork, emergent care, and public health/QI than in earlier periods. When transitioning from residency to fellowship, learners were always rated as needing greater supervision than they had been accorded at the end of residency and sometimes even more than they had been accorded at the start of residency. CONCLUSIONS: Although development over training is often imagined as continuous and monotonically increasing competence, this study provides empirical evidence supporting the idea that entrustment is a set of discrete decisions. The relaxation of supervision in training is not a linear process. Even with a seamless curriculum, supervision is tightly bound to the training setting. Several explanations for these findings are discussed.

Narrative Performance Level Assignments at Initial Entrustment and Graduation: Integrating EPAs and Milestones to Improve Learner Assessment.

PURPOSE: To determine which narrative performance level for each general pediatrics entrustable professional activity (EPA) reflects the minimum level clinical competency committees (CCCs) felt should be associated with graduation as well as initial entrustment and compare expected narrative performance levels (ENPLs) for each EPA with actual narrative performance levels (ANPLs) assigned to residents at initial entrustment. METHOD: A series of 5 narratives, corresponding to the 5 milestone performance levels, were developed for each of the 17 general pediatrics EPAs. In academic year (AY) 2015-2016, the CCCs at 22 Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network member sites reported ENPLs for initial entrustment and at time of graduation. From AYs 2015-2016 to 2017-2018, programs reported ANPLs for initial entrustment decisions. ENPLs and ANPLs were compared using a logistic mixed effects model. RESULTS: ENPLs for graduation and entrustment were most often level 3 (competent) followed by level 4 (proficient). For 8 EPAs, the ENPLs for graduation and entrustment were the same. For the remaining 9, some programs would entrust residents before graduation or graduate them before entrusting them. There were 4,266 supervision level reports for initial entrustment for which an ANPL was provided. ANPLs that were lower than the ENPLs were significantly more likely to be assigned to the medical home-well child (OR = 0.39; 95% CI: 0.26-0.57), transition to adult care (OR = 0.43; 95% CI: 0.19-0.95), behavioral or mental health (OR = 0.36; 95% CI: 0.18-0.71), make referrals (OR = 0.31; 95% CI: 0.17-0.55), lead a team (OR = 0.34; 95% CI: 0.22-0.52), and handovers (OR = 0.18; 95% CI: 0.09-0.36) EPAs. CONCLUSIONS: CCCs reported lower ENPLs for graduation than for entrustment for 5 EPAs, possibly indicating curricular gaps that milestones and EPAs could help identify.

Discovering the Benefits of Group Peer Review of Submitted Manuscripts.

Problem: Traditionally, journal editors expect individuals to complete peer reviews of submitted manuscripts on their own. Recently, a number of editors of health sciences journals have begun to support, and even espouse, the practice of group peer review (GPR). With GPR, multiple individuals work together to complete the review with permission from the journal editor. Motivated by the idea that GPR could provide a meaningful service learning experience for participants in an interprofessional educational scholarship course, we conducted three such reviews and subsequently reflected on our experience and the lessons we learned. We frame our reflections using guiding principles from the domains of peer review, professional development, and educational scholarship. Intervention: The course director arranged for manuscripts to review with the editors of three health sciences journals. Each GPR occurred during a separate weekly session of the course. Each GPR was completed using a similar set of steps, which included (a) gaining familiarity with review criteria, (b) reading aloud and discussing the manuscript's abstract as a class, (c) reading and critiquing assigned sections as individuals and then small groups, (d) building consensus and sharing notes, (e) having the course director synthesize notes into a single review for submission to the journal. Context: The course on educational scholarship involved 15 faculty representing faculty from the University of Utah's School of Medicine, College of Nursing, College of Pharmacy, College of Health, and School of Dentistry. The course director led three GPR sessions mid-way through the yearlong course. Impact: Participants' reflections indicate that GPR (a) conformed to principles of effective peer review; (b) resulted in a meaningful service learning experience within a formal professional development program, deepening understanding of core concepts of educational scholarship; and (c) represented an authentic example of engaging in educational scholarship (i.e., designing and evaluating an intervention while drawing upon and contributing to a body of shared understanding within a community of practice). Lessons Learned: Our principles-based approach to completing GPR within a professional development course on educational scholarship can serve as a model for others to follow. A rigorous, meaningful group review can occur in 1 hour using a combination of group and individual activities focused on matching review criteria to the submitted manuscript. As a result, we continue to include GPR in future offerings of this interprofessional course on educational scholarship, and we continue to study ways to optimize its value as a service learning experience.

Crossing the Gap: Using Competency-Based Assessment to Determine Whether Learners Are Ready for the Undergraduate-to-Graduate Transition.

In 2011, the Education in Pediatrics Across the Continuum (EPAC) Study Group recruited four medical schools (University of California, San Francisco; University of Colorado; University of Minnesota; and University of Utah) and their associated pediatrics clerkship and residency program directors to be part of a consortium to pilot a model designed to advance learners from undergraduate medical education (UME) to graduate medical education (GME) and then to fellowship or practice based on competence rather than time spent in training. The central design features of this pilot included predetermined expectations of performance and transition criteria to ensure readiness to progress from UME to GME, using the Core Entrustable Professional Activities for Entering Residency (Core EPAs) as a common assessment framework. Using this framework, each site team (which included, but was not limited to, the EPAC course, pediatric clerkship, and pediatric residency program directors) monitored learners' progress, with the site's clinical competency committee marking the point of readiness to transition from UME to GME (i.e., the attainment of supervision level 3a). Two of the sites implemented time-variable transition from UME to GME, based on when a learner met the performance expectations and transition criteria. In this Article, the authors describe each of the four sites' implementation of Core EPA assessment and their approach to gathering the data necessary to determine readiness for transition. They conclude by offering recommendations and lessons learned from the pilot's first seven years of development, adaptation, and implementation of assessment strategies across the sites, and discussing next steps.

Retrospective evaluation of Project Envision: A community mobilization pilot program to prevent sexual violence in New York City.

Sexual violence is a public health problem associated with short- and long-term physical and mental health consequences. Most interventions that aim to prevent sexual violence before it occurs target individual-level change or promote bystander training. Community-level interventions, while increasingly recommended in the sexual violence prevention field, are rarely documented in peer-reviewed literature. This paper is a targeted process evaluation of Project Envision, a 6-year pilot initiative to address social norms at the root of sexual violence through coalition building and community mobilization in three New York City neighborhoods, and reflects the perspectives of those charged with designing and implementing the program. Evaluation methods included a systematic literature review, archival source document review, and key informant interviews. Three themes emerged from the results: community identity and implications for engagement; capacity and readiness for community mobilization and consequences for implementation; and impacts on participants. Lessons learned include the limitations of using geographic boundaries to structure community interventions in urban settings; carefully considering whether communities should be mobilized around an externally-identified issue; translating theoretical frameworks into concrete tasks; assessing all coalition partners and organizations for readiness; critically evaluating available resources; and recognizing that community organizing is a skill that requires investment from funders. We conclude that Project Envision showed promise for shifting institutional norms towards addressing root causes of sexual violence in addition to providing victim services.

Phonological awareness and language intervention in preschoolers from low socio-economic backgrounds: a longitudinal investigation.

This study examines the literacy outcomes for children from socially disadvantaged backgrounds who had received specific whole-class phonological awareness (PA) and language intervention in preschool. The participants were 57 children who had been involved in the original intervention study. Their PA skills, letter-sound knowledge, real word and non-word spelling and reading comprehension were assessed in Grade 2. The results indicated that children who had received intervention in preschool performed similarly to the children who had not received intervention. The gains made in PA and language skills post intervention had failed to augment further literacy development. A post hoc examination of individual student profiles, however, revealed that a subgroup of children who had received intervention had maintained their enhanced performance and that the intervention cohort had similar scores on tests of PA ability to their age-matched peers in the population. It was concluded that whole-class, teacher-delivered, PA and language intervention, while effective in the short term, does not lead to a generalized improvement in literacy skills in Grade 2. Possible reasons for the failure of the program to produce medium term gains are discussed.