Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment.

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Pediatric Emergency Medicine Fellowship Program Directors' Viewpoint: Minimum Levels of Entrustment for Graduating Fellows and Practicing Physicians to Perform the Subspecialty's Professional Activities.

OBJECTIVES: Entrustable Professional Activities (EPAs) are essential tasks physicians perform within their professions. Entrustment levels that pediatric emergency medicine (PEM) fellowship program directors (FPDs) expect graduating fellows to achieve for PEM-specific and common pediatric subspecialty EPAs remain unreported. This study aims to determine minimum entrustment levels FPDs require fellows to achieve to graduate from fellowship and to compare FPD expectations for fellows versus practicing PEM physicians. METHODS: Secondary analysis of PEM-specific data from a national multispecialty cross-sectional survey of pediatric subspecialty FPDs. For 6 PEM-specific and 7 common pediatric subspecialty EPAs, PEM FPDs indicated (1) minimum entrustment levels fellows should achieve by training completion, (2) whether they would allow a fellow to graduate below these minimum levels, and (3) minimum levels for safe and effective practice by PEM physicians. Minimum levels were defined as the level that more than 80% of FPDs would not drop below. RESULTS: Sixty of 77 PEM FPDs (78%) completed the survey. Most respondents did not require fellows to achieve the highest level (level 5-no supervision) by graduation for any PEM-specific EPAs. The median level FPDs expected for practicing PEM physicians was 5 (trusted to perform without supervision) for EPAs 1 and 4 and level 4 (indirect supervision for complex cases) for the remaining PEM-specific EPAs. Minimum levels expected by FPDs for common subspecialty pediatric EPAs were lower for both groups. CONCLUSIONS: Most PEM FPDs indicated that they would graduate fellows before their achievement of the highest entrustment level for all EPAs. Most also indicated that they do not expect practicing PEM physicians to perform all EPAs without supervision. These findings indicate need for stakeholders to evaluate current structure and outcomes of PEM fellowship programs and for institutions and organizations to ensure adequate support in time and resources for ongoing learning for practicing PEM physicians.

Fellow Entrustment for the Common Pediatric Subspecialty Entrustable Professional Activities Across Subspecialties.

OBJECTIVE: To determine the relationship between level of supervision (LOS) ratings for the Common Pediatric Subspecialty Entrustable Professional Activities (EPAs) with their associated subcompetency milestones across subspecialties and by fellowship training year. METHODS: Clinical Competency Committees (CCCs) in 14 pediatric subspecialties submitted LOS ratings for 6 Common Subspecialty EPAs and subcompetency milestone levels mapped to these EPAs. We examined associations between these subcompetency milestone levels and LOS ratings across subspecialty training year by fitting per-EPA linear mixed effects models, regressing LOS rating on milestone level and on training year. RESULTS: CCCs from 211 pediatric fellowship programs provided data for 369 first, 336 second, and 331 third year fellows. Mean subcompetency milestone levels increased similarly among subspecialties for most EPAs compared with the reference, Adolescent Medicine. Mean subcompetency milestones mapped to each EPA and mean EPA LOS ratings generally increased by training year across all subspecialties. CONCLUSIONS: Subcompetency milestones levels mapped to each Common Subspecialty EPA and the EPA LOS ratings increase similarly across subspecialties and by training year, providing validity evidence for using EPA LOS to assess pediatric subspecialty trainee performance. This study supports the development of tools to facilitated the CCC evaluation process across all pediatric subspecialties.

Continued Supervision for the Common Pediatric Subspecialty Entrustable Professional Activities May Be Needed Following Fellowship Graduation.

PURPOSE: Entrustable professional activities (EPAs) are one approach to competency-based medical education (CBME), and 7 EPAs have been developed that address content relevant for all pediatric subspecialties. However, it is not known what level of supervision fellowship program directors (FPDs) deem necessary for graduation. The Subspecialty Pediatrics Investigator Network (SPIN) investigated FPD perceptions of the minimum level of supervision required for a trainee to successfully graduate. METHOD: In 2017, SPIN surveyed all FPDs of accredited fellowships for 14 subspecialties. For each EPA, the minimum supervision level for graduation (ranging from observation only to unsupervised practice) was set such that no more than 20% of FPDs would accept a lower level. RESULTS: The survey response rate was 82% (660/802). The minimum supervision level for graduation varied across the 7 EPAs from 2 (direct) to 4 (indirect for complex cases), with significant differences between EPAs. The percentage of FPDs desiring a lower minimum supervision level ranged from 3% to 17%. Compared with the 4 nonclinical EPAs (quality improvement, management, lead within the profession, scholarship), the 3 clinical EPAs (consultation, handover, lead a team) had higher minimum supervision graduation levels (P < .001), with less likelihood that an FPD would graduate a learner below their minimum level (P < .001). CONCLUSIONS: Consensus among FPDs across all pediatric subspecialties demonstrates the potential need for ongoing supervision for graduates in all 7 common pediatric subspecialty EPAs after fellowship. As CBME programs are implemented, processes and infrastructure to support new graduates are important considerations for leaders.

Maintenance of Certification Pediatrics: Pediatric Emergency Medicine (PEM): The New Part 3 Maintenance of Certification Assessment Option.

ABSTRACT: Starting in 2022, the American Board of Pediatrics will launch the Maintenance of Certification Assessment for Pediatrics: Pediatric Emergency Medicine (MOCA-Peds: PEM) longitudinal assessment, which will provide an at-home alternative to the point-in-time examination. This longitudinal assessment will help engage PEM physicians participating in continuing certification in a more flexible and continuous lifelong, self-directed learning process while still providing a summative assessment of their knowledge. This commentary provides background information on MOCA-Peds and an introduction to MOCA-Peds: PEM and how it gives the PEM physician another option to participate in continuing certification.

Emergency Medicine Providers' Knowledge and Management of Pediatric Tropical Diseases: A Needs Assessment.

OBJECTIVE: The aim of this study was to perform a needs assessment of pediatric (PEM) and general emergency medicine (EM) provider knowledge, comfort, and current practice patterns in the evaluation of pediatric tropical infectious diseases. METHODS: An online survey was developed based on educational priorities identified by an expert panel via modified Delphi methodology. The survey included assessment of providers' typical evaluation, diagnosis, and treatment of tropical diseases and was distributed to PEM and EM providers in 2 large professional organizations. RESULTS: A total of 333 physicians (285 PEM, 32 EM, 8 combined PEM/EM, and 8 general pediatricians in emergency department) participated. Fifty-five percent of vignettes were answered correctly. Those who trained outside the United States or Canada (odds ratio [OR], 1.8; 95% confidence interval [CI], 1.1-3.0) and PEM-trained providers (OR, 2.6; 95% CI, 1.2-5.9) were more likely to answer questions correctly. Providers answered more questions correctly about dengue (76%) and tuberculosis (77%) than typhoid (53%) and malaria (39%) (OR, 3.8; 95% CI, 3.0-4.9). Diagnostic evaluation for tropical diseases was variable with greater than 75% agreement for only 2 tests: blood smears in febrile patients from Africa (86%) and bacterial stool cultures in patients with bloody stools from Africa, Asia, or Latin America (94%). Providers had low (62%) or medium (35%) comfort level with pediatric tropical diseases, and 93% were interested in accessing emergency department-specific resources. CONCLUSIONS: Pediatric EM and EM providers' knowledge and evaluation for pediatric tropical diseases are variable. Providers recognized their knowledge gaps and expressed interest in gaining access to resources and guidelines to standardize and improve evaluation and treatment of these diseases.

Validity of Level of Supervision Scales for Assessing Pediatric Fellows on the Common Pediatric Subspecialty Entrustable Professional Activities.

PURPOSE: Entrustable professional activities (EPAs) represent the routine and essential activities that physicians perform in practice. Although some level of supervision scales have been proposed, they have not been validated. In this study, the investigators created level of supervision scales for EPAs common to the pediatric subspecialties and then examined their validity in a study conducted by the Subspecialty Pediatrics Investigator Network (SPIN). METHOD: SPIN Steering Committee members used a modified Delphi process to develop unique scales for six of the seven common EPAs. The investigators sought validity evidence in a multisubspecialty study in which pediatric fellowship program directors and Clinical Competency Committees used the scales to evaluate fellows in fall 2014 and spring 2015. RESULTS: Separate scales for the six EPAs, each with five levels of progressive entrustment, were created. In both fall and spring, more than 300 fellows in each year of training from over 200 programs were assessed. In both periods and for each EPA, there was a progressive increase in entrustment levels, with second-year fellows rated higher than first-year fellows (P < .001) and third-year fellows rated higher than second-year fellows (P < .001). For each EPA, spring ratings were higher (P < .001) than those in the fall. Interrater reliability was high (Janson and Olsson's iota = 0.73). CONCLUSIONS: The supervision scales developed for these six common pediatric subspecialty EPAs demonstrated strong validity evidence for use in EPA-based assessment of pediatric fellows. They may also inform the development of scales in other specialties.

Essentials of Pediatric Emergency Medicine Fellowship: Part 6: Program Administration.

This article is the sixth in a 7-part series that aims to comprehensively describe the current state and future directions of pediatric emergency medicine (PEM) fellowship training from the essential requirements to considerations for successfully administering and managing a program to the careers that may be anticipated upon program completion. This article provides a broad overview of administering and supervising a PEM fellowship program. It explores 3 topics: the principles of program administration, committee management, and recommendations for minimum time allocated for PEM fellowship program directors to administer their programs.

Essentials of PEM Fellowship Part 2: The Profession in Entrustable Professional Activities.

This article is the second in a 7-part series that aims to comprehensively describe the current state and future directions of pediatric emergency medicine (PEM) fellowship training from the essential requirements to considerations for successfully administering and managing a program to the careers that may be anticipated upon program completion. This article describes the development of PEM entrustable professional activities (EPAs) and the relationship of these EPAs with existing taxonomies of assessment and learning within PEM fellowship. It summarizes the field in concepts that can be taught and assessed, packaging the PEM subspecialty into EPAs.

Creation and Assessment of a Bad News Delivery Simulation Curriculum for Pediatric Emergency Medicine Fellows.

Background  Bad news in the context of health care has been broadly defined as significant information that negatively alters people's perceptions of the present or future. Effectively delivering bad news (DBN) in the setting of the emergency department requires excellent communication skills. Evidence shows that bad news is frequently given inadequately. Studies show that trainees need to devote more time to developing this skill through formalized training. This program's objectives were to utilize trained standardized patients in a simulation setting to assist pediatric emergency medicine (PEM) fellows in the development of effective, sensitive, and compassionate communication with patients and family members when conveying bad news, and to recognize and respond to the patient/parent's reaction to such news. Methods PEM fellows participated in a novel curriculum utilizing simulated patients (SPs) acting as the patient's parent and immersive techniques in a realistic and supportive environment. A baseline survey was conducted to ascertain participant demographics and previous experience with simulation and DBN. Experienced, multi-disciplinary faculty participated in a training workshop with the SPs one week prior to course delivery. Three scenarios were developed for bad news delivery. Instructors watched via remote video feed while the fellows individually interacted with the SPs and then participated in a confidential debriefing. Fellows later joined for group debriefing. Fellow characteristics, experience, and self-perceived comfort pre/post-course were collected.   Results Baseline data demonstrated that 78% of fellows reported DBN two or more times per month. Ninety-three percent of fellows in this study were present during the delivery of news about the death of a child to a parent or family member in the six-month period preceding this course. Fellows' self-reported comfort level in DBN to a patient/family and dealing with patient and parent emotions improved significantly (p=0.034 and p=0.046, respectively). Conclusions Pediatric emergency medicine fellows frequently deliver bad news. A course using SPs was well received by trainees and resulted in improvement in self-assessed skills and comfort. This curriculum provides the opportunity for fellows to receive patient/parent feedback of their communication skills and observations from skilled instructors. This methodology should be considered when creating training curricula for bad news delivery skills.

Essentials of Pediatric Emergency Medicine Fellowship: Part 1: An Overview.

This article is the first in a 7-part series (Table 1) that aims to comprehensively describe the current state and future directions of pediatric emergency medicine fellowship training from the essential requirements to considerations for successfully administering and managing a program to the careers that may be anticipated on program completion. This overview article provides a framework for the series.

A Novel Approach to Combining Pediatric Emergency Medicine and Global Health Fellowships.

OBJECTIVES: To describe the creation of the first known combined Pediatric Emergency Medicine-Global Health (PEM-GH) fellowship for graduates of pediatric or emergency medicine residency programs. METHODS: We detail the necessary infrastructure for a successfully combined PEM-GH fellowship including goals, objectives, curriculum, timeline, and funding. The fellowship is jointly supported by the department of pediatrics, section of pediatric emergency medicine (PEM), and the hospital. Fellows complete all requirements for the PEM fellowship and Global Health, the latter requiring an additional 12 months of training. Components of the Global Health fellowship include international fieldwork, scholarly activity abroad, advanced degree coursework, disaster training, and didactic curricula. RESULTS: Since 2005, 9 fellows (8 pediatric-trained and 1 emergency medicine-trained) have completed or are enrolled in the PEM-GH fellowship; 3 have graduated. All fellows have completed or are working toward advanced degrees and have or will participate in the disaster management course. Fellows have had 7 presentations at national or international meetings and have published 6 articles in peer-reviewed journals. Of the three graduates, all are working in academic PEM-GH programs and work internationally in Africa and/or Latin America. CONCLUSIONS: Our response to a global trend toward improvement in PEM care was the development of the first combined PEM-GH fellowship program. Recognizing the value of this program within our own institution, we now offer it as a model for building such programs in the future. This fellowship program promises to be a paradigm that can be used nationally and internationally, and it establishes a foundation for a full-fledged accredited and certified subspecialty.

Development and assessment of a pediatric emergency medicine simulation and skills rotation: meeting the demands of a large pediatric clerkship.

OBJECTIVE: To implement a curriculum using simulation and skills training to augment a Pediatric Emergency Medicine (PEM) rotation within a pediatric clerkship. BACKGROUND: PEM faculty are often challenged with a high learner to teacher ratio in a chaotic clinical setting. This challenge was heightened when our pediatric clerkship's traditional 1-week PEM rotation (consisting of 4 students completing four 8-hour ED shifts/week) expanded to 8 students every 2 weeks. We sought to meet this challenge by integrating simulation-based education into the rotation. METHODS: Clerkship students from March to June 2012 completed our traditional rotation. Students between July and October 2012 completed the new PEM-SIM curriculum with 19 hours ED shifts/week and 16 hours/week of simulation/skills training. Pre/post-tests evaluated 1) medical management/procedural comfort (five-point Likert scale); and 2) PEM knowledge (15 multiple-choice questions). RESULTS: One hundred and nine students completed the study (48 traditional, 61 PEM-SIM). Improvement in comfort was significantly higher for the PEM-SIM group than the traditional group for 6 of 8 (75%) medical management items (p<0.05) and 3 of 7 (43%) procedures, including fracture splinting, lumbar puncture, and abscess incision/drainage (p<0.05). PEM-SIM students had significantly more improvement in mean knowledge compared to the traditional group (p<0.001). CONCLUSIONS: We have successfully integrated 16 hours/week of faculty-facilitated simulation-based education into a PEM rotation within our clerkship. This curriculum is beneficial in clinical settings with high learner to teacher ratios and when patient care experiences alone are insufficient for all students to meet rotation objectives.

Investing in future pediatric subspecialists: a fellowship curriculum that prepares for the transition to academic careers.

BACKGROUND: The experience of transitioning to an academic faculty position can be improved with standardized educational interventions. Although a number of such interventions have been described, few utilize an evaluation framework, describe a robust evaluation process, and address why their interventions were successful. In this article, the authors apply a logic model to describe their efforts to develop, implement, evaluate, and revise a comprehensive academic career development curriculum among pediatric subspecialty fellows. They describe inputs, activities, outputs, and outcomes using quantitative data from fellow evaluations and qualitative data from faculty interviews. METHODS: Methods are described under the input and activities sections. The curriculum started with collaboration among educational leadership and conducting a needs assessment. Using the needs assessment results and targeted learning objectives, we piloted the curriculum and then implemented the full curriculum 1 year later. RESULTS: Results are described under the outputs and outcomes sections. We present immediate, short-term, and 6-month evaluation data. Cumulative data over 3 years reveal that fellows consistently acquired knowledge relevant to transitioning and that they applied acquired knowledge to prepare for finding jobs and career advancement. The curriculum also benefits faculty instructors who gain a sense of reward by filling a critical knowledge gap and fostering fellows' professional growth. CONCLUSION: The authors relate the success and effectiveness of the curriculum to principles of adult learning, and share lessons learned, including the importance of buy-in from junior and senior fellows and faculty, collaboration, and designating the time to teach and learn.