The APPD Longitudinal Educational Assessment Research Network's First Decade.

ABSTRACT: In 2009, the Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN), a national educational research network, was formed. We report on evaluation of the network after 10 years of operation by reviewing program context, input, processes, and products to measure its progress in performing educational research that advances training of future pediatricians. Historical changes in medical education shaped the initial development of the network. APPD LEARN now includes 74% (148 of 201) of US Pediatric residency programs and has recently incorporated a network of Pediatric subspecialty fellowship programs. At the time of this evaluation, APPD LEARN had approved 19 member-initiated studies and 14 interorganizational studies, resulting in 23 peer-reviewed publications, numerous presentations, and 7 archived sharable data sets. Most publications focused on how and when interventions work rather than whether they work, had high scores for reporting rigor, and included organizational and objective performance outcomes. Member program representatives had positive perceptions of APPD LEARN's success, with most highly valuing participation in research that impacts training, access to expertise, and the ability to make authorship contributions for presentations and publication. Areas for development and improvement identified in the evaluation include adopting a formal research prioritization process, infrastructure changes to support educational research that includes patient data, and expanding educational outreach within and outside the network. APPD LEARN and similar networks contribute to high-rigor research in pediatric education that can lead to improvements in training and thereby the health care of children.

Academic General Pediatrics Hiring Practices and Completion of Academic General Pediatrics Fellowships.

OBJECTIVE: Academic General Pediatrics (AGP) is a pediatric subspecialty with substantial faculty contributions in clinical care, research, education, and advocacy. However, AGP fellowship recruitment challenges exist. We aimed to describe AGP hiring practices from 2014 to 2019 and the role of fellowship training in hiring decisions. METHODS: We conducted a cross-sectional survey study of AGP Division Directors (DDs) and Fellowship Program Directors (PDs) from US-based academic institutions. Survey questions were developed iteratively and pilot-tested for content validity. Participants were identified from the Association of American Medical Colleges' directory of pediatric departments, Academic Pediatric Association's AGP Accreditation Committee's list of fellowship programs, and institutional websites. Descriptive analysis was used for close ended survey questions. Narrative responses were reviewed for trends. RESULTS: Forty-nine DDs (57%) and 22 PDs (73%) responded. All DDs reported at least one available faculty position and 73% reported filling a position with protected time. PDs reported 89 graduating fellows, 88% of whom secured an academic position with protected time. Seventy-percent of DDs and 100% of PDs reported that AGP fellows could secure an academic position with protected time, while only 22% and 1%, respectively, reported a graduating pediatric resident could secure a similar position. DDs indicated AGP fellowship trained candidates are preferable for enhancing research and education programs. CONCLUSION: AGP remains an active subspecialty and the majority of graduating fellows secured faculty positions with protected time. Further studies are needed to understand ways to improve visibility of AGP fellowships.

No Faculty Required: Use of a Health Literacy Low Inference Self-Assessment Measure to Promote Behavior Change.

OBJECTIVE: To determine if use of a health literacy low-inference, self-assessment measure (LISAM), promoted behavior change as measured by increased use of health literacy communication skills (HLCS). METHODS: The LISAM is a tool used by educators to self-assess their performances after giving a lecture. The tool is low inference because it self-assesses behaviors that are specific, with little room for subjectivity. Forty-four third-year medical students self-assessed HLCS using a LISAM modified to include health literacy communication skills (LISAM-HLCS).  Self-assessment followed participation in an audio recorded, standardized patient encounter and again after listening to the recording.  Students also created 3 written goals for improvement.  This session was repeated 1 week later. RESULTS: At Session 2, 71.4% of students met at least 2 of their 3 self-created objectives. The 3 most commonly created objectives were using teach-back, asking more open ended questions, and obtaining patient input into the management plan. Use of the LISAM increased HLCS use at Session 2 versus Session 1 as assessed by both students and study investigators (P < .05). CONCLUSIONS: Without faculty present, students met and adjusted objectives, catalyzing changes in HLCS. The LISAM-HLCS has the potential to empower students to improve communication skills and to reduce dependence on faculty observations.

An Annotated Bibliography of Key Studies in Medical Education in 2018: Applying the Current Literature to Pediatric Educational Practice and Scholarship.

Pediatric educators desire to and should strive to incorporate current educational methods and ideas into their professional practices. The overwhelming volume of medical education literature makes this difficult. This article provides an overview of 18 key articles from the 2018 literature that the authors considered impactful for the field of pediatric medical education. The author group has extensive combined leadership experience and expertise across the continuum of pediatric medical education and used an iterative, staged process to review 2270 abstracts from 13 medical education-related journals. This process aimed to identify a subset of articles that were most relevant to educational practice and scholarship and most applicable to pediatric medical education. Author pairs independently reviewed and scored abstracts and reached consensus to identify the abstracts that best met these criteria. Selected abstracts were discussed using different author pairs to determine the final articles included in this review. The 18 articles selected are summarized. The results showed a cluster of studies related to assessment, learner education and teaching, communication, and culture and climate. This review offers a summary for educators interested in remaining knowledgeable and current regarding the most relevant and valuable information in the field of medical education.

Longitudinal Assessment of Resident Performance Using Entrustable Professional Activities.

Importance: Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. Objective: To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. Design, Setting, and Participants: Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. Interventions: Longitudinal, prospective assessment using EPAs. Main Outcomes and Measures: Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. Results: Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). Conclusions and Relevance: This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence.

A theory-informed, process-oriented Resident Scholarship Program.

BACKGROUND: The Accreditation Council for Graduate Medical Education requires residency programs to provide curricula for residents to engage in scholarly activities but does not specify particular guidelines for instruction. We propose a Resident Scholarship Program that is framed by the self-determination theory (SDT) and emphasize the process of scholarly activity versus a scholarly product. METHODS: The authors report on their longitudinal Resident Scholarship Program, which aimed to support psychological needs central to SDT: autonomy, competence, and relatedness. By addressing those needs in program aims and program components, the program may foster residents' intrinsic motivation to learn and to engage in scholarly activity. To this end, residents' engagement in scholarly processes, and changes in perceived autonomy, competence, and relatedness were assessed. RESULTS: Residents engaged in a range of scholarly projects and expressed positive regard for the program. Compared to before residency, residents felt more confident in the process of scholarly activity, as determined by changes in increased perceived autonomy, competence, and relatedness. Scholarly products were accomplished in return for a focus on scholarly process. CONCLUSIONS: Based on our experience, and in line with the SDT, supporting residents' autonomy, competence, and relatedness through a process-oriented scholarship program may foster the curiosity, inquisitiveness, and internal motivation to learn that drives scholarly activity and ultimately the production of scholarly products.

The Pediatrics Milestones Assessment Pilot: Development of Workplace-Based Assessment Content, Instruments, and Processes.

PURPOSE: To report on the development of content and user feedback regarding the assessment process and utility of the workplace-based assessment instruments of the Pediatrics Milestones Assessment Pilot (PMAP). METHOD: One multisource feedback instrument and two structured clinical observation instruments were developed and refined by experts in pediatrics and assessment to provide evidence for nine competencies based on the Pediatrics Milestones (PMs) and chosen to inform residency program faculty decisions about learners' readiness to serve as pediatric interns in the inpatient setting. During the 2012-2013 PMAP study, 18 U.S. pediatric residency programs enrolled interns and subinterns. Faculty, residents, nurses, and other observers used the instruments to assess learner performance through direct observation during a one-month rotation. At the end of the rotation, data were aggregated for each learner, milestone levels were assigned using a milestone classification form, and feedback was provided to learners. Learners and site leads were surveyed and/or interviewed about their experience as participants. RESULTS: Across the sites, 2,338 instruments assessing 239 learners were completed by 630 unique observers. Regarding end-of-rotation feedback, 93% of learners (128/137) agreed the assessments and feedback "helped me understand how those with whom I work perceive my performance," and 85% (117/137) agreed they were "useful for constructing future goals or identifying a developmental path." Site leads identified several benefits and challenges to the assessment process. CONCLUSIONS: PM-based instruments used in workplace-based assessment provide a meaningful and acceptable approach to collecting evidence of learner competency development. Learners valued feedback provided by PM-based assessment.

Twelve tips for developing, implementing, and sustaining medical education fellowship programs: Building on new trends and solid foundations.

Medical education fellowship programs (MEFPs) are a form of faculty development contributing to an organization's educational mission and participants' career development. Building an MEFP requires a systematic design, implementation, and evaluation approach which aligns institutional and individual faculty goals. Implementing an MEFP requires a team of committed individuals who provide expertise, guidance, and mentoring. Qualified MEFP directors should utilize instructional methods that promote individual and institutional short and long term growth. Directors must balance the use of traditional design, implementation, and evaluation methodologies with advancing trends that may support or threaten the acceptability and sustainability of the program. Drawing on the expertise of 28 MEFP directors, we provide twelve tips as a guide to those implementing, sustaining, and/or growing a successful MEFP whose value is demonstrated by its impacts on participants, learners, patients, teaching faculty, institutions, the greater medical education community, and the population's health.

Blueprint for Action: Visioning Summit on the Future of the Workforce in Pediatrics.

The Federation of Pediatric Organizations engaged members of the pediatric community in an 18-month process to envision the future of the workforce in pediatrics, culminating in a Visioning Summit on the Future of the Workforce in Pediatrics. This article documents the planning process and methods used. Four working groups were based on the 4 domains that are likely to affect the future workforce: Child Health Research and Training, Diversity and Inclusion, Gender and Generations, and Pediatric Training Along the Continuum. These groups identified the issues and trends and prioritized their recommendations. Before the summit, 5 key megatrends cutting across all domains were identified:1. Aligning Education to the Emerging Health Needs of Children and Families 2. Promoting Future Support for Research Training and for Child Health Research 3. Striving Toward Mastery Within the Profession 4. Aligning and Optimizing Pediatric Practice in a Changing Health Care Delivery System 5. Taking Advantage of the Changing Demographics and Expertise of the Pediatric Workforce At the Visioning Summit, we assembled members of each of the working groups, the Federation of Pediatric Organizations Board of Directors, and several invited guests to discuss the 5 megatrends and develop the vision, solutions, and actions for each megatrend. Based on this discussion, we offer 10 recommendations for the field of pediatrics and its leading organizations to consider taking action.

Investing in future pediatric subspecialists: a fellowship curriculum that prepares for the transition to academic careers.

BACKGROUND: The experience of transitioning to an academic faculty position can be improved with standardized educational interventions. Although a number of such interventions have been described, few utilize an evaluation framework, describe a robust evaluation process, and address why their interventions were successful. In this article, the authors apply a logic model to describe their efforts to develop, implement, evaluate, and revise a comprehensive academic career development curriculum among pediatric subspecialty fellows. They describe inputs, activities, outputs, and outcomes using quantitative data from fellow evaluations and qualitative data from faculty interviews. METHODS: Methods are described under the input and activities sections. The curriculum started with collaboration among educational leadership and conducting a needs assessment. Using the needs assessment results and targeted learning objectives, we piloted the curriculum and then implemented the full curriculum 1 year later. RESULTS: Results are described under the outputs and outcomes sections. We present immediate, short-term, and 6-month evaluation data. Cumulative data over 3 years reveal that fellows consistently acquired knowledge relevant to transitioning and that they applied acquired knowledge to prepare for finding jobs and career advancement. The curriculum also benefits faculty instructors who gain a sense of reward by filling a critical knowledge gap and fostering fellows' professional growth. CONCLUSION: The authors relate the success and effectiveness of the curriculum to principles of adult learning, and share lessons learned, including the importance of buy-in from junior and senior fellows and faculty, collaboration, and designating the time to teach and learn.

Pediatricians and health literacy: descriptive results from a national survey.

OBJECTIVE: To describe pediatricians' self-reported experiences with health literacy, use of basic and enhanced communication techniques, and perceived barriers to effective communication during office visits. DESIGN/METHODS: A national, random sample of 1605 nonretired, posttraining American Academy of Pediatrics members were surveyed in 2007 about health literacy and patient communication as part of the Periodic Survey of Fellows. The response rate was 56% (N = 900). RESULTS: Eight-one percent of the pediatricians were aware of a situation in the previous 12 months in which a parent had not sufficiently understood health information that had been delivered to him or her. In addition, 44% of all pediatricians were aware of a communication-related error in patient care within the previous 12 months. Using simple language (99%), repeating key information (92%), and presenting only 2 or 3 concepts at a time (76%) were the most commonly used communication strategies. Enhanced communication techniques recommended by health literacy experts such as teach-back and indicating key points on written educational materials were used less often (23% and 28%, respectively). The most common reported barriers to effective communication were limited time to discuss information (73%), volume of information (65%), and complexity of information (64%). The majority of physicians rated themselves highly in their ability to identify caregiver understanding (64%), but only 21% rated themselves as very good or excellent in identifying a parent with a literacy problem. Fifty-seven percent of the respondents were interested in training to improve communication skills, and 58% reported that they would be very likely to use easy-to-read written materials, if available from the American Academy of Pediatrics. CONCLUSIONS: Pediatricians are aware of health literacy-related problems and the need for good communication with families but struggle with time demands to implement these skills. Despite awareness of communication-related errors in patient care, pediatricians report underutilizing enhanced techniques known to improve communication.