Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Virtual Interviews and the Pediatric Emergency Medicine Match Geography: A National Survey.

Introduction: Virtual interviews (VI) are now a permanent part of pediatric emergency medicine (PEM) recruitment, especially given the cost and equity advantages. Yet inability to visit programs in person can impact decision-making, leading applicants to apply to more programs. Moreover, the cost advantages of VI may encourage applicants to apply to programs farther away than they might otherwise have been willing or able to travel. This could create unnecessary strain on programs. We conducted this study to determine whether PEM fellowship applicants would apply to a larger number of programs and in different geographic patterns with VI (2020 and 2021) as compared to in-person interviews (2018 and 2019). Methods: We conducted an anonymous national survey of all PEM fellows comparing two cohorts: current fellows who interviewed inperson (applied in 2018/2019) and fellows who underwent VIs in 2020/2021 (current fellows and those recently matched in 2021). The study took place in March-April 2022. Questions focused on geographic considerations during interviews and the match. We used descriptive statistics, chi-square and t-tests for analysis. Results: Overall response rate was 42% (231/550); 32% (n = 74) interviewed in person and 68% (n = 157) virtually. Fellows applied to a median of 4/6 geographic regions (interquartile range 2, 5). Most applied for fellowship both in the same region as residency (216, 93%) and outside (192, 83%). Only the Pacific region saw a statistically significant increase in applicants during VI (59.9% vs 43.2%, P = 0.02). There was no statistical difference in the number of programs applied to during in-person vs VI (mean difference (95% confidence interval 0.72, -2.8 - 4.2). A majority matched in their preferred state both during VI (60.4%) and in-person interviews (65.7%). The difference was not statistically significant (P = 0.45). Conclusion: While more PEM fellowship applicants applied outside the geographic area where their residency was and to the Pacific region, there was no overall increase in the number of programs or geographic areas PEM applicants applied to during VI as compared to in-person interview seasons. As this was the first two years of VI, ongoing data collection will further identify trends and the impactof VI.

Selection and Recruitment Strategies among Competitive Pediatric Training Programs and the Impact of Diversity.

OBJECTIVE: Aspects of the written application, interview and ranking may negatively impact recruitment of underrepresented in medicine (URiM) applicants. Our objectives were to explore knowledge, attitudes, and perceptions of pediatric faculty who assess potential trainees and how diversity impacts these assessments. METHODS: We performed qualitative interviews of 20 geographically diverse faculty at large pediatric residencies and fellowships. We analyzed data using the constant comparative method to develop themes. RESULTS: Four main themes emerged. CONCLUSIONS: We describe ways in which bias infiltrates recruitment and strategies to promote diversity. Many strategies are variably implemented and the impact on workforce diversity in pediatric training programs remains unknown.

Identifying Gender and Racial Bias in Pediatric Fellowship Letters of Recommendation: Do Word Choices Influence Interview Decisions?

OBJECTIVES: To describe linguistic differences in letters of recommendation (LORs) for pediatric fellowship candidates based on applicant and letter writer demographics and to examine if these differences influenced the decision to interview a candidate for a fellowship position. STUDY DESIGN: LORs for applicants to 8 pediatric subspecialty fellowships at a single academic center from the 2020 Match were analyzed in this cross-sectional study. Frequency of validated agentic and communal terms in each letter were determined by a language processing web application. Bias was determined as having a >5% surplus of agentic or communal terms. RESULTS: We analyzed 1521 LORs from 409 applicants: 69% were women, 28% were under-represented minorities in medicine (URM), and 50% were invited to interview. Overall, 66% of LORs were agentic biased, 16% communal biased, and 19% neutral. There was no difference in bias in LORs by an applicant's gender (woman 67% agentic vs man 62% agentic; P = .058), race, or ethnicity (non-URM 65% agentic vs URM 67% agentic; P = .660). Despite a lower frequency of agentic terms in LORs for applicants invited for interviews, when accounting for other components of an application and applicant demographics, no significant association was made between language bias in LORs and fellowship interview status. CONCLUSIONS: The frequency of agentic and communal terms in LORs for pediatric subspecialty fellowship candidates were not found to influence the decision to invite a candidate to interview. However, raising awareness of potential areas of bias within the pediatric fellowship selection process might lead to a more equitable and holistic approach to application review.

Spotlighting the imbalance: Gender disparities among speakers and awardees at pediatric emergency medicine conferences.

BACKGROUND: There are wide variations in the gender makeup of speakers at national pediatric emergency medicine (PEM) conferences with no significant change in recent years. OBJECTIVE: Gender disparities exist among national speakers and award recipients. PEM represents the intersection of pediatrics, a female-dominated specialty with approximately 58% women, and emergency medicine, a male-dominated specialty. We describe the proportion of women speakers and award recipients at two national PEM conferences, the American Academy of Pediatrics (AAP) Section on Emergency Medicine (SOEM) and the Advanced PEM Assembly (APEMA), to the AAP National Conference & Exhibition (NCE), a national pediatric conference. METHODS: Data from SOEM and APEMA, obtained from 2016 to 2021 were compared to the 2021 NCE. Invited speakers, abstract presenters, and award recipients were identified. Gender was determined by searching each individual's name for self-identification. Gender proportions were compared across conferences, speaker type, and year. RESULTS: Compared to the NCE, a significantly smaller proportion of women were invited speakers at APEMA (NCE 59.9% vs. APEMA 38.8%, p < 0.001), but similar proportions of women were invited speakers (53.9%, p = 0.178) and awardees at SOEM (50% vs. 50%, p = 1.0). A larger number of women were SOEM abstract presenters than invited speakers (63.3% vs. 53.9%, p = 0.041). Between 2016 and 2021, the proportion of women invited speakers (SOEM, p = 0.744; APEMA, p = 0.947) or abstract presenters (SOEM, p = 0.632) did not significantly change. CONCLUSIONS: Compared to NCE, women are underrepresented as speakers at APEMA, but not at SOEM. Abstract presenters are more likely to be women compared to invited speakers. While awards appear equally distributed, recipients do not mirror the proportion of women in PEM. Conference organizers and leaders in PEM should ensure gender equity in national recognition.

Clinical competency committee perceptions of entrustable professional activities and their value in assessing fellows: A qualitative study of pediatric subspecialty program directors.

OBJECTIVES: To examine the composition and processes of Clinical Competency Committees (CCCs) assigning entrustable professional activity (EPA) levels of supervision for pediatric subspecialty fellows and to examine fellowship program director (FPD) perspectives about using EPAs to determine fellows' graduation readiness. METHODS: A qualitative study was performed using one-on-one interviews with a purposeful sample of pediatric subspecialty FPDs to yield a thematic analysis. Semi-structured interview guides were used for participants who self-identified as EPA users or non-users. Inductive analysis and coding were performed on transcripts until theoretical sufficiency was attained. RESULTS: Twenty-eight FPDs were interviewed. There was significant variability in the composition and processes of CCCs across subspecialties. FPDs felt that CCCs intuitively understand what entrustment means, allowing for ease of application of level of supervision (LOS) scales and consensus. FPDs perceived that EPAs provided a global assessment of fellows and are one tool to determine graduation readiness. CONCLUSIONS: Although there was variability in the makeup and processes of CCCs across subspecialties, FPDs believe EPAs are intuitive and relatively easy to implement. Consensus can be reached easily using EPA-specific LOS scales focusing on entrustment. FPDs desire a better understanding of how EPAs should be used for graduation.

Pediatric Emergency Medicine Fellowship Program Directors' Viewpoint: Minimum Levels of Entrustment for Graduating Fellows and Practicing Physicians to Perform the Subspecialty's Professional Activities.

OBJECTIVES: Entrustable Professional Activities (EPAs) are essential tasks physicians perform within their professions. Entrustment levels that pediatric emergency medicine (PEM) fellowship program directors (FPDs) expect graduating fellows to achieve for PEM-specific and common pediatric subspecialty EPAs remain unreported. This study aims to determine minimum entrustment levels FPDs require fellows to achieve to graduate from fellowship and to compare FPD expectations for fellows versus practicing PEM physicians. METHODS: Secondary analysis of PEM-specific data from a national multispecialty cross-sectional survey of pediatric subspecialty FPDs. For 6 PEM-specific and 7 common pediatric subspecialty EPAs, PEM FPDs indicated (1) minimum entrustment levels fellows should achieve by training completion, (2) whether they would allow a fellow to graduate below these minimum levels, and (3) minimum levels for safe and effective practice by PEM physicians. Minimum levels were defined as the level that more than 80% of FPDs would not drop below. RESULTS: Sixty of 77 PEM FPDs (78%) completed the survey. Most respondents did not require fellows to achieve the highest level (level 5-no supervision) by graduation for any PEM-specific EPAs. The median level FPDs expected for practicing PEM physicians was 5 (trusted to perform without supervision) for EPAs 1 and 4 and level 4 (indirect supervision for complex cases) for the remaining PEM-specific EPAs. Minimum levels expected by FPDs for common subspecialty pediatric EPAs were lower for both groups. CONCLUSIONS: Most PEM FPDs indicated that they would graduate fellows before their achievement of the highest entrustment level for all EPAs. Most also indicated that they do not expect practicing PEM physicians to perform all EPAs without supervision. These findings indicate need for stakeholders to evaluate current structure and outcomes of PEM fellowship programs and for institutions and organizations to ensure adequate support in time and resources for ongoing learning for practicing PEM physicians.

Pediatric Residency Preparedness for Pediatric Emergency Medicine Fellowship.

OBJECTIVE: The aim of the study was to examine the perception of pediatric emergency medicine (PEM) program directors (PDs), associate PDs (APDs) and first-year fellows regarding the preparedness of new PEM fellows who have graduated from pediatric residency programs. METHODS: Unique surveys were created and electronically distributed to PEM PDs/APDs and first-year PEM fellows. Individual and institutional demographic information was collected. Using a 5-point Likert scale, survey items centered the perceived preparedness of first-year PEM fellows within 5 domains: professionalism, independence/autonomy, psychomotor skills, clinical evaluation and management, and academia/scholarship. RESULTS: Forty percent (48/119) of eligible PDs and APDs and 56% (82/147) of eligible first-year PEM fellows responded. Most PDs/APDs strongly agreed or agreed that incoming fellows perform adequately in areas of professionalism, independence/autonomy, and clinical evaluation and management. The PDs/APDs perceived fellows to be less prepared in the academia/scholarship domain and several psychomotor skills. Most first-year PEM fellows perceived themselves as prepared in areas of professionalism and clinical evaluation and management. Fellows had varied feelings of preparedness in the domains of independence/autonomy, psychomotor skills, and academia/scholarship. Overall, most PDs/APDs (54%) and fellows (84%) feel that pediatric residency training was strong or very strong. CONCLUSIONS: Most respondents in both groups felt that in general, pediatric residency programs adequately train residents for PEM fellowship. Both groups felt that the strengths of general pediatric training were among the domains of professionalism and clinical evaluation and management, whereas psychomotor skills and academia and scholarship were areas of improvement. These findings may be used by general pediatric residency and PEM fellowship programs to guide curriculum development.

Guiding Fellows to Independent Practice: Current Trends in Pediatric Emergency Medicine Fellow Supervision.

BACKGROUND: Recent studies highlight the importance of physician readiness to practice beyond graduate training. The Accreditation Council for Graduate Medical Education mandates that pediatric emergency medicine (PEM) fellows be prepared for independent practice by allowing "progressive responsibility for patient care." Prior unpublished surveys of program directors (PDs) indicate variability in approaches to provide opportunities for more independent practice during fellowship training. OBJECTIVES: The aims of the study were to describe practices within PEM fellowship programs allowing fellows to work without direct supervision and to identify any barriers to independent practice in training. DESIGN/METHODS: An anonymous electronic survey of PEM fellowship PDs was performed. Survey items were developed using an iterative modified Delphi process and pilot tested. Close-ended survey responses and demographic variables were summarized with descriptive statistics. Responses to open-ended survey items were reviewed and categorized by theme. RESULTS: Seventy two of 84 PDs (88%) responded to the survey; however, not all surveys were completed. Of the 68 responses to whether fellows could work without direct supervision (as defined by the Accreditation Council for Graduate Medical Education) during some part of their training, 31 (45.6%) reported that fellows did have this opportunity. In most programs, clinical independence was conditional on specific measures including the number of clinical hours completed, milestone achievement, and approval by the clinical competency committee. Reported barriers to fellow practice without direct oversight included both regulatory and economic constraints. CONCLUSIONS: Current training practices that provide PEM fellows with conditional clinical independence are variable. Future work should aim to determine best practices of entrustment, identify ideal transition points, and mitigate barriers to graduated responsibility.

Can Behavior-Based Interviews Reduce Bias in Fellowship Applicant Assessment?

PURPOSE: Components of trainee applications may introduce bias based on race or gender. Behavior-based interviews (BBIs) rely on structured questions to elicit applicants' past experiences to predict future behavior. Our objective was to implement BBIs in one fellowship program and compare applicant assessment by race and gender when using a standardized assessment tool versus a BBI-based tool. METHODS: In 2019 and 2020, we developed BBIs and BBI-specific assessments; 6 of 15 faculty were trained in this interview method. Applicants completed 6 interviews with either a BBI or unstructured format. All faculty completed a standardized assessment on applicants. BBI faculty also completed a BBI-specific assessment. Normalized average scores were calculated and used to rank applicants into quartiles. Race was categorized into White, underrepresented minorities (URMs; Black and Hispanics), and non-URMs (all others). Faculty and applicants were surveyed about BBIs. RESULTS: Seventy-five applicants were interviewed. Significant differences were found in standardized assessment scores (White 1.01 +/- 0.09, non-URM 1.02 +/- 0.08, URM .94 +/- 0.07; P = .02) and quartiles by race (P = .05), but not for BBI scores (White 0.98 +/- 0.09, non-URM 1.03 +/- 0.09, URM 1.02 +/- 0.1; P = .18) or quartiles by race (P = .17). There were no significant differences in score or quartile by gender for either tool. The majority of faculty and applicant survey respondents commented positively about BBIs. CONCLUSION: BBIs were successfully implemented and generally reviewed positively by faculty and applicants. BBIs reduced racial differences in applicant assessments. Applicant assessment may benefit from structured tools to mitigate potential biases.

The Impact of Behavioral Anchors in the Assessment of Fellowship Applicants: Reducing Rater Biases.

INTRODUCTION: No standardized evaluation tool for fellowship applicant assessment exists. Assessment tools are subject to biases and scoring tendencies which can skew scores and impact rankings. We aimed to develop and evaluate an objective assessment tool for fellowship applicants. METHODS: We detected rater effects in our numerically scaled assessment tool (NST), which consisted of 10 domains rated from 0 to 9. We evaluated each domain, consolidated redundant categories, and removed subjective categories. For 7 remaining domains, we described each quality and developed a question with a behaviorally-anchored rating scale (BARS). Applicants were rated by 6 attendings. Ratings from the NST in 2018 were compared with the BARS from 2020 for distribution of data, skewness, and inter-rater reliability. RESULTS: Thirty-four applicants were evaluated with the NST and 38 with the BARS. Demographics were similar between groups. The median score on the NST was 8 out of 9; scores <5 were used in less than 1% of all evaluations. Distribution of data was improved in the BARS tool. In the NST, scores from 6 of 10 domains demonstrated moderate skewness and 3 high skewness. Three of the 7 domains in the BARS showed moderate skewness and none had high skewness. Two of 10 domains in the NST vs 5 of 7 domains in the BARS achieved good inter-rater reliability. CONCLUSION: Replacing a standard numeric scale with a BARS normalized the distribution of data, reduced skewness, and enhanced inter-rater reliability in our evaluation tool. This provides some validity evidence for improved applicant assessment and ranking.

Virtual Interviewing for Pediatric Emergency Medicine Fellowship-A National Survey.

OBJECTIVE: This study aimed to evaluate both applicant and interviewer satisfaction with the virtual interviewing process for pediatric emergency medicine (PEM) fellowship in hopes to improve the fellowship interviewing process. It was proposed that fellowship programs and applicants would prefer virtual interviews over traditional interviews. METHODS: A survey developed in collaboration with UT Southwestern PEM fellowship leaders and national PEM leaders was sent to all PEM fellowship applicants and programs at the conclusion of the 2020 interview season and rank list submission. The applicant survey obtained information on ease of virtual interviews and whether applicants felt that they obtained adequate information from virtual interviews to make informed program selections. Program director surveys collected data on thoughts and feelings about virtual interviews and obstacles encountered during the recruitment season. Both surveys asked about costs for interviews and interview type preference. RESULTS: A response rate of 49% from applicants and 47% from programs was obtained. Virtual interview days were similar in the amount of time and staff hours used compared with traditional days. Applicants spent less on virtual interviews compared with those who underwent traditional interviews (average $725 vs $4312). Programs received more applications than the prior year and spent less money during the virtual cycle. The majority of the applicants (90%) were comfortable with the virtual interview platform, and most (66%) agreed that virtual interviews provided adequate information to determine program rank. Geography was the number 1 rank determining factor. Programs and applicants preferred a form of in-person interviews. CONCLUSIONS: Virtual interviews provide cost savings for both applicants and programs. Despite this, both parties prefer a form of in-person interviews.

An evidence-based approach to nontraumatic ocular complaints in children.

Children commonly present to emergency departments with eye complaints in the absence of antecedent trauma. Signs and symptoms of ocular disease are often nonspecific. Red, swollen, or painful eyes may represent benign or vision-threatening processes, making recognition and triage challenging for the emergency clinician. This issue reviews the presentations of common nontraumatic ocular complaints and provides evidence-based recommendations for management in the emergency department.

A Community Emergency Department Rotation for Pediatric Emergency Medicine Fellows: Development and Evaluation.

OBJECTIVE: There have been increasing job opportunities for pediatric emergency medicine (PEM) specialists within community emergency departments (CEDs), where 90% of children receive emergency care. The majority of training for PEM fellows occurs in academic children's hospitals, which may expose fellows to different resources than CEDs. Our goals were to develop and to evaluate a CED rotation aimed at improving exposure to this care environment for PEM fellows. METHODS: Kern's 6 steps of curriculum development served as our framework to identify the problem, conduct a needs assessment, develop goals and objectives, formulate an educational strategy, and then plan the implementation and evaluation. Discussion and buy-in among trainees and hospital leadership took place during rotation development. Informal and web-based surveys of PEM fellows and CED attendings evaluated the rotation and assessed participants' reactions, changes in knowledge, and changes in behavior. RESULTS: The CED rotation was piloted at 1 PEM fellowship program over 2 years. Nine PEM fellows participated in the month-long rotation. The majority of participants reported that the rotation goals and objectives were met and that they acquired new skills and rated the CED rotation as good or excellent. Challenges to the CED rotation included lower patient acuity and distance to the CED. CONCLUSIONS: A new CED rotation was developed, piloted, and well received by participants. Given the potential differences in practice between tertiary care children's emergency departments and CEDs, as well as increasing job opportunities in CEDs, program directors should consider incorporating a CED rotation for their PEM fellows.

Physiologic monitoring practices during pediatric procedural sedation: a report from the Pediatric Sedation Research Consortium.

OBJECTIVES: To describe the frequency of different physiologic monitoring modalities and combinations of modalities used during pediatric procedural sedation; to describe how physiologic monitoring varies among different classes of patients, health care providers (ie, ranging from anesthesiologists to emergency medicine physicians to nurse practitioners), procedures, and sedative medications employed; and to determine the proportion of sedations meeting published guidelines for physiologic monitoring. DESIGN This was a prospective, observational study from September 1, 2007, through March 31, 2011. SETTING: Data were collected in areas outside of the operating room, such as intensive care units, radiology, emergency departments, and clinics. PARTICIPANTS: Thirty-seven institutions comprise the Pediatric Sedation Research Consortium that prospectively collects data on procedural sedation/anesthesia performed outside of the operating room in all children up to age 21 years. MAIN OUTCOME MEASURES: Data including demographics, procedure performed, provider level, adverse events, medications, and physiologic monitors used are entered into a web-based system. RESULTS: Data from 114 855 subjects were collected and analyzed. The frequency of use of each physiologic monitoring modality by health care provider type, medication used, and procedure performed varied significantly. The largest difference in frequency of monitoring use was seen between providers using electrocardiography (13%-95%); the smallest overall differences were seen in monitoring use based on the American Society of Anesthesiologists classifications (1%-10%). Guidelines published by the American Academy of Pediatrics, the American College of Emergency Physicians, and the American Society of Anesthesiologists for nonanesthesiologists were adhered to for 52% of subjects. CONCLUSIONS: A large degree of variability exists in the use of physiologic monitoring modalities for pediatric procedural sedation. Differences in monitoring are evident between sedation providers, medications, procedures, and patient types.