Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment.

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Spanish-Speaking Caregivers' Perception of Emergency Physicians' Interpersonal and Communication Skills in a Pediatric Emergency Department.

OBJECTIVES: Medical communication is more than just the delivery of information; language differences between physicians and patients/caregivers create a challenge to providing effective care in the pediatric emergency department (ED). Overcoming this barrier is vital to providing high-quality care. We evaluated Spanish- versus English-speaking caregivers' perception of their pediatric ED physicians' interpersonal and communication skills. We also compared perceptions of Spanish- versus English-speaking caregivers who self-identified as Hispanic. METHODS: This study is a retrospective analysis of data from surveys administered in an urban, free-standing children's hospital ED. Surveys were administered in English and Spanish to pediatric patient caregivers. In person, video, and telephonic interpretations were available during patient encounters. RESULTS: There were 2542 (82.4%) surveys completed in English and 543 (17.6%) in Spanish. There were significant differences in demographic data of English versus Spanish survey respondents, including level of education, insurance status, and rates of nonpublic insurance. Spanish survey respondents rated their physicians' interpersonal skills lower than English survey respondents. There were 1455 (47%) surveys completed by the respondents who self-identified as Hispanic. Within this group, 928 (63.8%) respondents completed the survey in English and 527 (36.2%) in Spanish. Among this Hispanic population, the Spanish survey respondents rated their physicians' interpersonal and communication skills lower than English survey respondents. After adjusting for education level and insurance type, these differences persisted. CONCLUSIONS: Language barriers have a meaningful impact on physician ability to communicate effectively in the pediatric ED. Improving physicians' ability to overcome this barrier is essential toward enriching patient outcomes and experience in the ED.

Pediatric Emergency Medicine Fellowship Program Directors' Viewpoint: Minimum Levels of Entrustment for Graduating Fellows and Practicing Physicians to Perform the Subspecialty's Professional Activities.

OBJECTIVES: Entrustable Professional Activities (EPAs) are essential tasks physicians perform within their professions. Entrustment levels that pediatric emergency medicine (PEM) fellowship program directors (FPDs) expect graduating fellows to achieve for PEM-specific and common pediatric subspecialty EPAs remain unreported. This study aims to determine minimum entrustment levels FPDs require fellows to achieve to graduate from fellowship and to compare FPD expectations for fellows versus practicing PEM physicians. METHODS: Secondary analysis of PEM-specific data from a national multispecialty cross-sectional survey of pediatric subspecialty FPDs. For 6 PEM-specific and 7 common pediatric subspecialty EPAs, PEM FPDs indicated (1) minimum entrustment levels fellows should achieve by training completion, (2) whether they would allow a fellow to graduate below these minimum levels, and (3) minimum levels for safe and effective practice by PEM physicians. Minimum levels were defined as the level that more than 80% of FPDs would not drop below. RESULTS: Sixty of 77 PEM FPDs (78%) completed the survey. Most respondents did not require fellows to achieve the highest level (level 5-no supervision) by graduation for any PEM-specific EPAs. The median level FPDs expected for practicing PEM physicians was 5 (trusted to perform without supervision) for EPAs 1 and 4 and level 4 (indirect supervision for complex cases) for the remaining PEM-specific EPAs. Minimum levels expected by FPDs for common subspecialty pediatric EPAs were lower for both groups. CONCLUSIONS: Most PEM FPDs indicated that they would graduate fellows before their achievement of the highest entrustment level for all EPAs. Most also indicated that they do not expect practicing PEM physicians to perform all EPAs without supervision. These findings indicate need for stakeholders to evaluate current structure and outcomes of PEM fellowship programs and for institutions and organizations to ensure adequate support in time and resources for ongoing learning for practicing PEM physicians.

Career Development in Pediatric Emergency Medicine: What Do We Need?

OBJECTIVES: The Women in Pediatric Emergency Medicine (PEM) subcommittee of the American Academy of Pediatrics Section on Emergency Medicine identified 2 top priorities for 2021: career development and mentorship/sponsorship. The objective of this study was to catalog and delineate the career development domains for women physicians in PEM. METHODS: After a review of the literature to identify the key areas for gaps for women in PEM, we used Q sort methodology to elicit domains for this subcommittee to address by survey of a national sample. RESULTS: One hundred fourteen discrete potential areas of interest for career development were identified by the working group based on salient themes from the literature and personal experiences. Forty-one Women in PEM subcommittee members (27%) completed the survey. The career development topics were sorted into the domains of personal (40.4%; n = 46), administrative (28.1%; n = 32), research (10.5%; n = 12), teaching (10.5%; n = 12), service (7.0%; n = 8), and clinical (3.5%; n = 4). CONCLUSIONS: This study demonstrates that the career development needs of women in PEM include a range of personal, teaching, research, administrative, clinical, and service domains. However, more than two-thirds of the career development topics were categorized into just 2 domains, administrative and personal.

Fellow Entrustment for the Common Pediatric Subspecialty Entrustable Professional Activities Across Subspecialties.

OBJECTIVE: To determine the relationship between level of supervision (LOS) ratings for the Common Pediatric Subspecialty Entrustable Professional Activities (EPAs) with their associated subcompetency milestones across subspecialties and by fellowship training year. METHODS: Clinical Competency Committees (CCCs) in 14 pediatric subspecialties submitted LOS ratings for 6 Common Subspecialty EPAs and subcompetency milestone levels mapped to these EPAs. We examined associations between these subcompetency milestone levels and LOS ratings across subspecialty training year by fitting per-EPA linear mixed effects models, regressing LOS rating on milestone level and on training year. RESULTS: CCCs from 211 pediatric fellowship programs provided data for 369 first, 336 second, and 331 third year fellows. Mean subcompetency milestone levels increased similarly among subspecialties for most EPAs compared with the reference, Adolescent Medicine. Mean subcompetency milestones mapped to each EPA and mean EPA LOS ratings generally increased by training year across all subspecialties. CONCLUSIONS: Subcompetency milestones levels mapped to each Common Subspecialty EPA and the EPA LOS ratings increase similarly across subspecialties and by training year, providing validity evidence for using EPA LOS to assess pediatric subspecialty trainee performance. This study supports the development of tools to facilitated the CCC evaluation process across all pediatric subspecialties.

Design of a point-of-care ultrasound curriculum for pediatric emergency medicine fellows: A Delphi study.

OBJECTIVES: There has been a steady increase in the growth and utilization of point-of-care ultrasound (POCUS) in pediatric emergency medicine (PEM). POCUS has been established as an Accreditation Council for Graduate Medical Education (ACGME) core requirement for accreditation of PEM fellowship programs. Despite this requirement, training guidelines regarding POCUS knowledge and skills have yet to be developed. The purpose of this project was to develop a curriculum and a competency checklist for PEM fellow POCUS education. METHODS: We formed a core leadership group based on expertise in one or more key areas: PEM, POCUS, curriculum development, or Delphi methods. We recruited 29 PEM POCUS or ultrasound education experts from North America to participate in a three-round electronic Delphi project. The first Delphi round asked experts to generate a list of the core POCUS knowledge and skills that a PEM fellow would need during training to function as an autonomous practitioner. Subsequent rounds prioritized the list of knowledge and skills, and the core leadership group organized knowledge and skills into global competencies and subcompetencies. RESULTS: The first Delphi round yielded 61 POCUS areas of knowledge and skills considered important for PEM fellow learning. After two subsequent Delphi rounds, the list of POCUS knowledge and skills was narrowed to 38 items that addressed elements of six global competencies. The core leadership group then revised items into subcompetencies and categorized them under global competencies, developing a curriculum that defined the scope (depth of content) and sequence (order of teaching) of these POCUS knowledge and skill items. CONCLUSIONS: This expert, consensus-generated POCUS curriculum provides detailed guidance for PEM fellowships to incorporate POCUS education into their programs. Our curriculum also identifies core ultrasound knowledge and skills needed by PEM fellows to perform the specific POCUS applications recommended in prior publications.

Achieving Entrustable Professional Activities During Fellowship.

BACKGROUND AND OBJECTIVES: Entrustable Professional Activities (EPAs) were developed to assess pediatric fellows. We previously showed that fellowship program directors (FPDs) may graduate fellows who still require supervision. How this compares with their expectations for entrustment of practicing subspecialists is unknown. METHODS: We surveyed US FPDs in 14 pediatric subspecialties through the Subspecialty Pediatrics Investigator Network between April and August 2017. For each of 7 common pediatric subspecialty EPAs, we compared the minimum level of supervision that FPDs required for graduation with the level they expected of subspecialists for safe and effective practice using the Friedman rank sum test and paired t test. We compared differences between subspecialties using linear regression. RESULTS: We collected data from 660 FPDs (response rate 82%). For all EPAs, FPDs did not require fellows to reach the level of entrustment for graduation that they expected of subspecialists to practice (P < .001). FPDs expected the least amount of supervision for the EPAs consultation and handovers. Mean differences between supervision levels for graduation and practice were smaller for clinical EPAs (consultation, handovers, lead a team) when compared with nonclinical EPAs (quality improvement, management, lead the profession and scholarship; P = .001) and were similar across nearly all subspecialties. CONCLUSIONS: Fellowship graduates may need continued development of clinical and nonclinical skills in their early practice period, underscoring a need for continued assessment and mentoring. Graduation readiness must be based on clear requirements, with alignment of FPD expectations and regulatory standards, to ensure quality care for patients.

Development of a novel pediatric point-of-care ultrasound question bank using a modified Delphi process.

BACKGROUND AND OBJECTIVES: Measuring pediatric emergency medicine (PEM) fellow competency in point-of-care ultrasound (POCUS) is important for ensuring adequate training and performance. Assessment may include direct observation, image review, quality assessment, and written examination. The purpose of this study was to develop a pediatric POCUS question bank that could subsequently be used as a POCUS assessment for graduating PEM fellows. METHODS: We organized a 10-person question writing group (QWG). Eight hold expertise in POCUS and two hold expertise in medical education. Members of the QWG created questions within four domains: interpretation/diagnosis (50% of questions), anatomy (30%), physics (10%), and pitfalls (10%). POCUS faculty ascertained content validity and the medical education faculty revised questions for syntax and readability. In 2016, we recruited 31 pediatric POCUS experts. The majority were members of the P2 Network, an international group of experts and leaders in PEM POCUS, to participate in three iterative rounds of a modified Delphi process to review, revise, and establish consensus on the question bank. RESULTS: Thirty-one pediatric POCUS experts participated in the three rounds of the modified Delphi process and evaluated 437 questions developed by the expert panel. Forty-nine percent (n = 216) of the questions were accepted in round 1, 30% (n = 130) in round 2, and 11% (n = 47) in the final round. The final question bank included 393 questions covering 17 pediatric POCUS applications. CONCLUSION: We developed a 393-question bank to aid in the assessment of PEM POCUS competency. Future work includes piloting the questions with PEM fellows to evaluate the response process and implementing the assessment tool to establish a minimum passing score.

Continued Supervision for the Common Pediatric Subspecialty Entrustable Professional Activities May Be Needed Following Fellowship Graduation.

PURPOSE: Entrustable professional activities (EPAs) are one approach to competency-based medical education (CBME), and 7 EPAs have been developed that address content relevant for all pediatric subspecialties. However, it is not known what level of supervision fellowship program directors (FPDs) deem necessary for graduation. The Subspecialty Pediatrics Investigator Network (SPIN) investigated FPD perceptions of the minimum level of supervision required for a trainee to successfully graduate. METHOD: In 2017, SPIN surveyed all FPDs of accredited fellowships for 14 subspecialties. For each EPA, the minimum supervision level for graduation (ranging from observation only to unsupervised practice) was set such that no more than 20% of FPDs would accept a lower level. RESULTS: The survey response rate was 82% (660/802). The minimum supervision level for graduation varied across the 7 EPAs from 2 (direct) to 4 (indirect for complex cases), with significant differences between EPAs. The percentage of FPDs desiring a lower minimum supervision level ranged from 3% to 17%. Compared with the 4 nonclinical EPAs (quality improvement, management, lead within the profession, scholarship), the 3 clinical EPAs (consultation, handover, lead a team) had higher minimum supervision graduation levels (P < .001), with less likelihood that an FPD would graduate a learner below their minimum level (P < .001). CONCLUSIONS: Consensus among FPDs across all pediatric subspecialties demonstrates the potential need for ongoing supervision for graduates in all 7 common pediatric subspecialty EPAs after fellowship. As CBME programs are implemented, processes and infrastructure to support new graduates are important considerations for leaders.

Maintenance of Certification Pediatrics: Pediatric Emergency Medicine (PEM): The New Part 3 Maintenance of Certification Assessment Option.

ABSTRACT: Starting in 2022, the American Board of Pediatrics will launch the Maintenance of Certification Assessment for Pediatrics: Pediatric Emergency Medicine (MOCA-Peds: PEM) longitudinal assessment, which will provide an at-home alternative to the point-in-time examination. This longitudinal assessment will help engage PEM physicians participating in continuing certification in a more flexible and continuous lifelong, self-directed learning process while still providing a summative assessment of their knowledge. This commentary provides background information on MOCA-Peds and an introduction to MOCA-Peds: PEM and how it gives the PEM physician another option to participate in continuing certification.

Impact of a Pediatric-Focused Communication Course on Patient/Caregiver-Perceived Physician Communication Skills in a Pediatric Emergency Department.

OBJECTIVES: To evaluate the impact of a pediatric-focused, relationship-centered communication course on patient/caregiver assessment of emergency department (ED) physician communication skills. METHODS: This was a prospective study; a previously validated assessment tool (Pediatric Physician Interpersonal Communication Skills Assessment [P-PICSA]) was used to collect pediatric patient/caregiver evaluation of ED physician communication skills. Emergency department physicians were required to attend a 5.5-hour communication course. Differences in precourse/postcourse P-PICSA and top-box scores were analyzed to determine course impact on ED physician communication skills. RESULTS: From July 2016 to August 2017, 49 pediatric ED physicians participated in the course, and 24 physicians (49%) had 6-month precourse/postcourse P-PICSA data. Postcourse scores were higher for all 13 items, with 3 items achieving statistical significance: (1) the doctor used words I could understand; (2) the way the doctor involved me in making decisions about my child's care in the ED; (3) how the doctor discussed next steps and/or follow-up plans for my child's care after we leave the ED (P < 0.05). Additionally, postcourse, top-box score percentages were higher for 12 items, with 1 item achieving statistical significance (P = 0.0386). CONCLUSIONS: An organization-wide, pediatric-focused, relationship-centered communication course improved patient/caregiver-perceived ED physician communication. Further research is necessary to assess long-term sustainability.

Agreement of Program Directors With Clinical Competency Committees for Fellow Entrustment.

OBJECTIVES: Fellowship program directors (FPD) and Clinical Competency Committees (CCCs) both assess fellow performance. We examined the association of entrustment levels determined by the FPD with those of the CCC for 6 common pediatric subspecialty entrustable professional activities (EPAs), hypothesizing there would be strong correlation and minimal bias between these raters. METHODS: The FPDs and CCCs separately assigned a level of supervision to each of their fellows for 6 common pediatric subspecialty EPAs. For each EPA, we determined the correlation between FPD and CCC assessments and calculated bias as CCC minus FPD values for when the FPD was or was not a member of the CCC. In addition, we examined the effect of program size, FPD understanding of EPAs, and subspecialty on the correlations. Data were obtained in fall 2014 and spring 2015. RESULTS: A total of 1040 fellows were assessed in the fall and 1048 in the spring. In both periods and for each EPA, there was a strong correlation between FPD and CCC supervision levels (P < .001). The correlation was somewhat lower when the FPD was not a CCC member (P < .001). Overall bias in both periods was small. CONCLUSIONS: The correlation between FPD and CCC assignment of EPA supervision levels is strong. Although slightly weaker when the FPD is not a CCC member, bias is small, so this is likely unimportant in determining fellow entrustment level. The similar performance ratings of FPDs and CCCs support the validity argument for EPAs as competency-based assessment tools.

Focused Research Infrastructure for Postgraduate Pediatric Emergency Medicine Fellows Increases Dissemination of Scholarly Work.

BACKGROUND: Many fellows in clinically driven subspecialties may have difficulty completing and publishing their scholarly projects due to lack of prior experience in research, selection of projects that are difficult to complete during fellowship, or mentorship challenges. This may be particularly true in pediatric emergency medicine (PEM) because research time may be longitudinally integrated with clinical rotations, rather than blocked as is common in other subspecialties. We describe the creation and outcomes of a structured program to increase academic productivity of PEM fellows. METHODS: This was a retrospective cohort study of scholarly productivity (publications in peer-reviewed journals, presentation at national meetings) for PEM fellows over 17 years in one fellowship program, before and after the implementation of a structured program. We reviewed obstacles to publication for prior fellow projects when developing the curriculum. Our multifaceted program consisted of milestone development, four in-person committee meetings, and abstract and manuscript development workshops. We utilized existing faculty members, most of whom were junior faculty, as committee members. Our primary outcome was the percentage of fellows who were first authors for peer-reviewed publications for their fellowship projects. National conference presentations were the secondary outcome. RESULTS: Data for 76 PEM fellows were eligible for analysis: 44 (58%) before and 32 after programmatic implementation. There was a statistically significant increase in the percentage of fellows who published their studies (32% vs. 63%; odds ratio [OR] = 3.6, 95% confidence interval [CI] = 1.4 to 9.3) after programmatic implementation. There were no differences in conference presentations (45% vs. 63%; OR = 2.0, 95% CI = 0.8-5.1) after implementation. CONCLUSIONS: Utilizing a small group of existing, predominantly junior faculty members, we created a structured program that enhanced PEM fellows' scholarly productivity and increased publications. We believe that this model is sustainable for and generalizable to other PEM fellowship programs.

Emergency Medicine Providers' Knowledge and Management of Pediatric Tropical Diseases: A Needs Assessment.

OBJECTIVE: The aim of this study was to perform a needs assessment of pediatric (PEM) and general emergency medicine (EM) provider knowledge, comfort, and current practice patterns in the evaluation of pediatric tropical infectious diseases. METHODS: An online survey was developed based on educational priorities identified by an expert panel via modified Delphi methodology. The survey included assessment of providers' typical evaluation, diagnosis, and treatment of tropical diseases and was distributed to PEM and EM providers in 2 large professional organizations. RESULTS: A total of 333 physicians (285 PEM, 32 EM, 8 combined PEM/EM, and 8 general pediatricians in emergency department) participated. Fifty-five percent of vignettes were answered correctly. Those who trained outside the United States or Canada (odds ratio [OR], 1.8; 95% confidence interval [CI], 1.1-3.0) and PEM-trained providers (OR, 2.6; 95% CI, 1.2-5.9) were more likely to answer questions correctly. Providers answered more questions correctly about dengue (76%) and tuberculosis (77%) than typhoid (53%) and malaria (39%) (OR, 3.8; 95% CI, 3.0-4.9). Diagnostic evaluation for tropical diseases was variable with greater than 75% agreement for only 2 tests: blood smears in febrile patients from Africa (86%) and bacterial stool cultures in patients with bloody stools from Africa, Asia, or Latin America (94%). Providers had low (62%) or medium (35%) comfort level with pediatric tropical diseases, and 93% were interested in accessing emergency department-specific resources. CONCLUSIONS: Pediatric EM and EM providers' knowledge and evaluation for pediatric tropical diseases are variable. Providers recognized their knowledge gaps and expressed interest in gaining access to resources and guidelines to standardize and improve evaluation and treatment of these diseases.

Improved Team Performance During Pediatric Resuscitations After Rapid Cycle Deliberate Practice Compared With Traditional Debriefing: A Pilot Study.

INTRODUCTION: Simulation-based medical education (SBME) improves medical knowledge compared with no intervention. In traditional SBME, more time is spent debriefing than practicing skills. Rapid cycle deliberate practice (RCDP) simulation allows learners to practice skills repetitively, receive brief interspersed feedback, and has been shown to improve individual performance of resuscitation skills in simulation; it has not been compared with traditional simulation methods. OBJECTIVE: The aim of the study was to compare traditional and RCDP SBME. METHODS: Four pediatric resuscitation cases (3 for teaching and 1 for testing) were developed. For the RCDP arm, traditional cases were deconstructed into sequences of progressively difficult rounds. The last RCDP round served as the traditional arm scenario.Learners received 1 type of instruction on 2 separate days. Pretest and posttest performance during simulation were video recorded and scored using the Simulation Team Assessment Tool; satisfaction surveys were collected. RESULTS: Pretest team performance was similar in both groups. Simulation Team Assessment Tool score improvement for RCDP was 7.2% (95% confidence interval, 3.4% to 11%) and traditional was 0.8% (95% confidence interval, -11% to 13%). The difference in improvement of the human factors subscore was statistically significant; RCDP improved 10.2% and traditional improved 1.7% (P = 0.013). The RCDP technique was well received by learners but caused fatigue. CONCLUSIONS: This pilot study showed a trend toward greater improvement in team performance and significantly greater improvement for human factors with RCDP compared with traditional simulation. Future studies comparing RCDP with other methods are needed to identify best practices and applications of RCDP, including which learners and learning objectives are best suited to RCDP.

Validity of Level of Supervision Scales for Assessing Pediatric Fellows on the Common Pediatric Subspecialty Entrustable Professional Activities.

PURPOSE: Entrustable professional activities (EPAs) represent the routine and essential activities that physicians perform in practice. Although some level of supervision scales have been proposed, they have not been validated. In this study, the investigators created level of supervision scales for EPAs common to the pediatric subspecialties and then examined their validity in a study conducted by the Subspecialty Pediatrics Investigator Network (SPIN). METHOD: SPIN Steering Committee members used a modified Delphi process to develop unique scales for six of the seven common EPAs. The investigators sought validity evidence in a multisubspecialty study in which pediatric fellowship program directors and Clinical Competency Committees used the scales to evaluate fellows in fall 2014 and spring 2015. RESULTS: Separate scales for the six EPAs, each with five levels of progressive entrustment, were created. In both fall and spring, more than 300 fellows in each year of training from over 200 programs were assessed. In both periods and for each EPA, there was a progressive increase in entrustment levels, with second-year fellows rated higher than first-year fellows (P < .001) and third-year fellows rated higher than second-year fellows (P < .001). For each EPA, spring ratings were higher (P < .001) than those in the fall. Interrater reliability was high (Janson and Olsson's iota = 0.73). CONCLUSIONS: The supervision scales developed for these six common pediatric subspecialty EPAs demonstrated strong validity evidence for use in EPA-based assessment of pediatric fellows. They may also inform the development of scales in other specialties.

Pediatric Emergency Medicine Online Curriculum Improves Resident Knowledge Scores, But Will They Use It?

OBJECTIVE: Shift work on a pediatric emergency medicine (PEM) rotation makes didactic scheduling difficult, thereby limiting teaching opportunities. These constraints make this rotation an ideal setting to supplement resident education with an online curriculum. We aimed to determine if implementation of an online curriculum during a resident PEM rotation improves posttest performance and increases satisfaction with resident educational experience. METHODS: This was a prospective before/after study of pediatric and emergency medicine residents on a 1-month rotation in a tertiary care pediatric emergency department. A curriculum was developed consisting of 17 online modules. In the first 5 months of the study, 42 control residents received traditional bedside teaching. In the last 12 months, 80 intervention residents completed at least 8 modules during their rotation. Both groups completed a pretest at rotation start and a posttest and end-of-rotation survey at rotation end. RESULTS: Control group pretest and posttest scores were not significantly different. In the intervention group, posttest scores were significantly increased compared with pretest scores (68 vs 59, P < 0.01). A low percentage of residents completed the study. Only 42% of the 189 residents enrolled in the intervention group completed the posttest and 28% completed the survey. CONCLUSIONS: Implementing an online PEM curriculum significantly improved knowledge. As residency programs face new duty hour requirements, online curricula may provide an effective way to supplement teaching. However, to capitalize on this self-directed curriculum, the low participation rates in this study suggest we must first determine and establish ways to overcome barriers to online learning.

Essentials of Pediatric Emergency Medicine Fellowship: Part 6: Program Administration.

This article is the sixth in a 7-part series that aims to comprehensively describe the current state and future directions of pediatric emergency medicine (PEM) fellowship training from the essential requirements to considerations for successfully administering and managing a program to the careers that may be anticipated upon program completion. This article provides a broad overview of administering and supervising a PEM fellowship program. It explores 3 topics: the principles of program administration, committee management, and recommendations for minimum time allocated for PEM fellowship program directors to administer their programs.