Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Validation of the PediBIRN-7 clinical prediction rule for pediatric abusive head trauma.

BACKGROUND: The PediBIRN-7 clinical prediction rule incorporates the (positive or negative) predictive contributions of completed abuse evaluations to estimate abusive head trauma (AHT) probability after abuse evaluation. Applying definitional criteria as proxies for AHT and non-AHT ground truth, it performed with sensitivity 0.73 (95 % CI: 0.66-0.79), specificity 0.87 (95 % CI: 0.82-0.90), and ROC-AUC 0.88 (95 % CI: 0.85-0.92) in its derivation study. OBJECTIVE: To validate the PediBIRN-7's AHT prediction performance in a novel, equivalent, patient population. PARTICIPANTS AND SETTINGS: Consecutive, acutely head-injured children <3 years hospitalized for intensive care across eight sites between 2017 and 2020 with completed skeletal surveys and retinal exams (N = 342). METHODS: Secondary analysis of an existing, cross-sectional, prospective dataset, including assignment of patient-specific estimates of AHT probability, calculation of AHT prediction performance measures (ROC-AUC, sensitivity, specificity, predictive values), and completion of sensitivity analyses to estimate best- and worst-case prediction performances. RESULTS: Applying the same definitional criteria, the PediBIRN-7 performed with sensitivity 0.74 (95 % CI: 0.66-0.81), specificity 0.77 (95 % CI: 0.70-0.83), and ROC-AUC 0.83 (95 % CI: 0.78-0.88). The reduction in ROC-AUC was statistically insignificant (p = .07). Applying physicians' final consensus diagnoses as proxies for AHT and non-AHT ground truth, the PediBIRN-7 performed with sensitivity 0.73 (95 % CI: 0.66-0.79), specificity 0.87 (95 % CI: 0.82-0.90), and ROC-AUC 0.90 (95 % CI: 0.87-0.94). Sensitivity analyses demonstrated minimal changes in rule performance. CONCLUSION: The PediBIRN-7's overall AHT prediction performance has been validated in a novel, equivalent, patient population. Its patient-specific estimates of AHT probability can inform physicians' AHT-related diagnostic reasoning after abuse evaluation.

Epidemiology of brief resolved unexplained events and impact of clinical practice guidelines in general and pediatric emergency departments.

OBJECTIVES: The aim of this study was to describe the incidence of brief resolved unexplained events (BRUEs) and compare the impact of a national clinical practice guideline (CPG) on admission and diagnostic testing practices between general and pediatric emergency departments (EDs). METHODS: Using the Nationwide Emergency Department Sample for 2012-2019, we conducted a cross-sectional study of children <1 year of age with an International Classification of Diseases diagnostic code for BRUE. Population incidence rate was estimated using Centers for Disease Control and Prevention birth data. ED incidence rate was estimated for all ED encounters. We used interrupted time series to evaluate the associated impact of the CPG publication on the outcomes of ED disposition (discharge, admission, and transfer) and electrocardiogram (ECG) use. RESULTS: Of 133,972 encounters for BRUE, 80.0% occurred in general EDs. BRUE population incidence was 4.28 per 1000 live births and the annual incidence remained stable (p = 0.19). BRUE ED incidence was 5.06 per 1000 infant ED encounters (p = 0.14). The impact of the BRUE CPG on admission rates was limited to pediatric EDs (level shift -23.3%, p = 0.002). Transfers from general EDs did not change with the CPG (level shift 2.2%, p = 0.17). After the CPG was published, ECGs increased by 13.7% in pediatric EDs (p = 0.005) but did not change in general EDs (level shift -0.2%, p = 0.82). CONCLUSIONS: BRUEs remain a common pediatric problem at a population level and in EDs. Although a disproportionate number of infants present to general EDs, there is differential uptake of the CPG recommendations between pediatric and general EDs. These findings may support quality improvement opportunities aimed at improving care for these infants and decreasing unnecessary hospital admissions or transfers.

Analysis of fellowship program director opinions of entrustable professional activities in adolescent medicine fellowship.

OBJECTIVES: This study aimed to explore the minimum entrustable professional activity (EPA) supervision levels at which pediatric fellowship program directors (FPDs) would be willing to graduate fellows and the levels deemed necessary for safe and effective practice for each of the common pediatric subspecialty and the four adolescent medicine-specific EPAs. METHODS: This cross-sectional study utilized survey data from pediatric FPDs in 2017. FPDs indicated the minimum level of supervision (LOS) for fellows at graduation and for safe and effective practice. RESULTS: 82 percent (23/28) of adolescent medicine FPDs completed the survey. For each EPA, there were differences (p<0.05) between LOS expected for graduation and for safe and effective practice. There was also variability in the level at which FPDs would graduate fellows. CONCLUSIONS: This study summarizes pediatric FPD opinions regarding the minimum levels of supervision required for fellows at the time of graduation as well as the levels deemed necessary for safe and effective practice. The difference between the minimum LOS at which FPDs would graduate a fellow and that deemed appropriate for safe and effective practice, along with variability in minimum LOS for graduation, highlight the need for clearer standards for fellowship graduation as well as more structured early career support for ongoing learning. These data highlight variability in FPD opinion regarding such expectations and both the need to better define desired training outcomes and potential need for post-graduation supervision in clinical practice.

Exploring Factors for Implementation of EPAs in Pediatric Subspecialty Fellowships: A Qualitative Study of Program Directors.

OBJECTIVE: To understand fellowship program directors' (FPDs) perspectives on facilitators and barriers to using entrustable professional activities (EPAs) in pediatric subspecialty training. METHODS: We performed a qualitative study of FPDs, balancing subspecialty, program size, geographic region and current uses of EPAs. A study coordinator conducted 1-on-1 interviews using a semistructured approach to explore EPA use or nonuse and factors supporting or preventing their use. Investigators independently coded transcribed interviews using an inductive approach and the constant comparative method. Group discussion informed code structure development and refinement. Iterative data collection and analysis continued until theoretical sufficiency was achieved, yielding a thematic analysis. RESULTS: Twenty-eight FPDs representing 11 pediatric subspecialties were interviewed, of whom 16 (57%) reported current EPA use. Five major themes emerged: (1) facilitators including the intuitive nature and simple wording of EPAs; (2) barriers such as workload burden and lack of a regulatory requirement; (2) variable knowledge and training surrounding EPAs, leading to differing levels of understanding; (3) limited current use of EPAs, even among self-reported users; and (4) complementary nature of EPAs and milestones. FPDs acknowledged the differing strengths of both EPAs and milestones but sought additional knowledge about the value added by EPAs for assessing trainees, including the impact on outcomes. CONCLUSIONS: Identified themes can inform effective and meaningful EPA implementation strategies: Supporting and educating FPDs, ongoing assessment of the value of EPAs in training, and practical integration with current workflow. Generating additional data and engaging stakeholders is critical for successful implementation for the pediatric subspecialties.

Retinal hemorrhage variation in inertial versus contact head injuries.

BACKGROUND: Abusive head trauma (AHT) is frequently accompanied by dense/extensive retinal hemorrhages to the periphery with or without retinoschisis (complex retinal hemorrhages, cRH). cRH are uncommon without AHT or major trauma. OBJECTIVE: The study objectives were to determine whether cRH are associated with inertial vs. contact mechanisms and are primary vs. secondary injuries. PARTICIPANTS AND SETTING: This retrospective study utilized a de-identified PediBIRN database of 701 children <3-years-old presenting to intensive care for head trauma. Children with motor vehicle related trauma and preexisting brain abnormalities were excluded. All had imaging showing head injury and a dedicated ophthalmology examination. METHODS: Contact injuries included craniofacial soft tissue injuries, skull fractures and epidural hematoma. Inertial injuries included acute impairment or loss of consciousness and/or bilateral and/or interhemispheric subdural hemorrhage. Abuse was defined in two ways, by 1) predetermined criteria and 2) caretaking physicians/multidisciplinary team's diagnostic consensus. RESULTS: PediBIRN subjects with cRH frequently experienced inertial injury (99.4 % (308/310, OR = 53.74 (16.91-170.77)) but infrequently isolated contact trauma (0.6 % (2/310), OR = 0.02 (0.0004-0.06)). Inertial injuries predominated over contact trauma among children with cRH sorted AHT by predetermined criteria (99.1 % (237/239), OR = 20.20 (6.09-67.01) vs 0.5 % (2/339), OR = 0.04 (0.01-0.17)). Fifty-nine percent of patients with cRH, <24 h altered consciousness, and inertial injuries lacked imaging evidence of brain hypoxia, ischemia, or swelling. CONCLUSIONS: cRH are significantly associated with inertial angular acceleration forces. They can occur without brain hypoxia, ischemia or swelling suggesting they are not secondary injuries.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment.

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Skeletal survey yields in low vs. high risk pediatric patients with skull fractures.

BACKGROUND: To assess for occult fractures, physicians often opt to obtain skeletal surveys (SS) in young, acutely head-injured patients who present with skull fractures. Data informing optimal decision management are lacking. OBJECTIVE: To determine the positive yields of radiologic SS in young patients with skull fractures presumed to be at low vs. high risk for abuse. PARTICIPANTS AND SETTING: 476 acutely head injured, skull-fractured patients <3 years hospitalized for intensive care across 18 sites between February 2011 and March 2021. METHODS: We conducted a retrospective, secondary analysis of the combined, prospective Pediatric Brain Injury Research Network (PediBIRN) data set. RESULTS: 204 (43 %) of 476 patients had simple, linear, parietal skull fractures. 272 (57 %) had more complex skull fracture(s). Only 315 (66 %) of 476 patients underwent SS, including 102 (32 %) patients presumed to be at low risk for abuse (patients who presented with a consistent history of accidental trauma; intracranial injuries no deeper than the cortical brain; and no respiratory compromise, alteration or loss of consciousness, seizures, or skin injuries suspicious for abuse). Only one of 102 low risk patients revealed findings indicative of abuse. In two other low risk patients, SS helped to confirm metabolic bone disease. CONCLUSIONS: Less than 1 % of low risk patients under three years of age who presented with simple or complex skull fracture(s) revealed other abusive fractures. Our results could inform efforts to reduce unnecessary skeletal surveys.

Clinical competency committee perceptions of entrustable professional activities and their value in assessing fellows: A qualitative study of pediatric subspecialty program directors.

OBJECTIVES: To examine the composition and processes of Clinical Competency Committees (CCCs) assigning entrustable professional activity (EPA) levels of supervision for pediatric subspecialty fellows and to examine fellowship program director (FPD) perspectives about using EPAs to determine fellows' graduation readiness. METHODS: A qualitative study was performed using one-on-one interviews with a purposeful sample of pediatric subspecialty FPDs to yield a thematic analysis. Semi-structured interview guides were used for participants who self-identified as EPA users or non-users. Inductive analysis and coding were performed on transcripts until theoretical sufficiency was attained. RESULTS: Twenty-eight FPDs were interviewed. There was significant variability in the composition and processes of CCCs across subspecialties. FPDs felt that CCCs intuitively understand what entrustment means, allowing for ease of application of level of supervision (LOS) scales and consensus. FPDs perceived that EPAs provided a global assessment of fellows and are one tool to determine graduation readiness. CONCLUSIONS: Although there was variability in the makeup and processes of CCCs across subspecialties, FPDs believe EPAs are intuitive and relatively easy to implement. Consensus can be reached easily using EPA-specific LOS scales focusing on entrustment. FPDs desire a better understanding of how EPAs should be used for graduation.

Diagnostic testing for and detection of physical abuse in infants with brief resolved unexplained events.

BACKGROUND: A Brief Resolved Unexplained Event (BRUE) can be a sign of occult physical abuse. OBJECTIVES: To identify rates of diagnostic testing able to detect physical abuse (head imaging, skeletal survey, and liver transaminases) at BRUE presentation. The secondary objective was to estimate the rate of physical abuse diagnosed at initial BRUE presentation through 1 year of age. PARTICIPANTS AND SETTING: Infants who presented with a BRUE at one of 15 academic or community hospitals were followed from initial BRUE presentation until 1 year of age for BRUE recurrence or revisits. METHODS: This study was part of the BRUE Research and Quality Improvement Network, a multicenter retrospective cohort examining infants with BRUE. Generalized estimating equations assessed associations with performance of diagnostic testing (adjusted odds ratio (aOR)). RESULTS: Of the 2036 infants presenting with a BRUE, 6.2 % underwent head imaging, 7.0 % skeletal survey, and 12.1 % liver transaminases. Infants were more likely to undergo skeletal survey if there were physical examination findings concerning for trauma (aOR 8.23, 95 % CI [1.92, 35.24], p < 0.005) or concerning social history (aOR 1.89, 95 % CI [1.13, 3.16], p = 0.015). There were 7 (0.3 %) infants diagnosed with physical abuse: one at BRUE presentation, one <3 days after BRUE presentation, and five >30 days after BRUE presentation. CONCLUSION: There were low rates of diagnostic testing and physical abuse identified in infants presenting with BRUE. Further study including standardized testing protocols is warranted to identify physical abuse in infants presenting with a BRUE.

Pediatric Emergency Medicine Fellowship Program Directors' Viewpoint: Minimum Levels of Entrustment for Graduating Fellows and Practicing Physicians to Perform the Subspecialty's Professional Activities.

OBJECTIVES: Entrustable Professional Activities (EPAs) are essential tasks physicians perform within their professions. Entrustment levels that pediatric emergency medicine (PEM) fellowship program directors (FPDs) expect graduating fellows to achieve for PEM-specific and common pediatric subspecialty EPAs remain unreported. This study aims to determine minimum entrustment levels FPDs require fellows to achieve to graduate from fellowship and to compare FPD expectations for fellows versus practicing PEM physicians. METHODS: Secondary analysis of PEM-specific data from a national multispecialty cross-sectional survey of pediatric subspecialty FPDs. For 6 PEM-specific and 7 common pediatric subspecialty EPAs, PEM FPDs indicated (1) minimum entrustment levels fellows should achieve by training completion, (2) whether they would allow a fellow to graduate below these minimum levels, and (3) minimum levels for safe and effective practice by PEM physicians. Minimum levels were defined as the level that more than 80% of FPDs would not drop below. RESULTS: Sixty of 77 PEM FPDs (78%) completed the survey. Most respondents did not require fellows to achieve the highest level (level 5-no supervision) by graduation for any PEM-specific EPAs. The median level FPDs expected for practicing PEM physicians was 5 (trusted to perform without supervision) for EPAs 1 and 4 and level 4 (indirect supervision for complex cases) for the remaining PEM-specific EPAs. Minimum levels expected by FPDs for common subspecialty pediatric EPAs were lower for both groups. CONCLUSIONS: Most PEM FPDs indicated that they would graduate fellows before their achievement of the highest entrustment level for all EPAs. Most also indicated that they do not expect practicing PEM physicians to perform all EPAs without supervision. These findings indicate need for stakeholders to evaluate current structure and outcomes of PEM fellowship programs and for institutions and organizations to ensure adequate support in time and resources for ongoing learning for practicing PEM physicians.

The association of subcortical brain injury and abusive head trauma.

BACKGROUND: Abusive head trauma (AHT) remains a major pediatric problem with diagnostic challenges. A small pilot study previously associated subcortical brain injury with AHT. OBJECTIVES: To investigate the association of subcortical injury on neuroimaging with the diagnosis of AHT. PARTICIPANTS AND SETTING: Children <3 years with acute TBI admitted to 18 PICUs between 2011 and 2021. METHODS: Secondary analysis of existing, combined, de-identified, cross-sectional dataset. RESULTS: Deepest location of visible injury was characterized as scalp/skull/epidural (n = 170), subarachnoid/subdural (n = 386), cortical brain (n = 170), or subcortical brain (n = 247) (total n = 973). Subcortical injury was significantly associated with AHT using both physicians' diagnostic impression (OR: 8.41 [95 % CI: 5.82-12.44]) and a priori definitional criteria (OR: 5.99 [95 % CI: 4.31-8.43]). Caregiver reports consistent with the child's gross motor skills and historically consistent with repetition decreased as deepest location of injury increased, p < 0.001. Patients with subcortical injuries were significantly more likely to have traumatic extracranial injuries such as rib fractures (OR 3.36, 95 % CI 2.30-4.92) or retinal hemorrhages (OR 5.97, 95 % CI 4.35-8.24), respiratory compromise (OR 12.12, 95 % CI 8.49-17.62), circulatory compromise (OR 6.71, 95 % CI 4.87-9.29), seizures (OR 3.18, 95 % CI 2.35-4.29), and acute encephalopathy (OR 12.44, 95 % CI 8.16-19.68). CONCLUSIONS: Subcortical injury is associated with a diagnosis of AHT, historical inaccuracies concerning for abuse, traumatic extracranial injuries, and increased severity of illness including respiratory and circulatory compromise, seizures, and prolonged loss of consciousness. Presence of subcortical injury should be considered as one component of the complex AHT diagnostic process.

Psychosis-Spectrum Screening and Assessment Within a College Counseling Center: A Pilot Study Exploring Feasibility and Clinical Need.

Evidence supports the use of brief psychosis-spectrum screening tools for identifying individuals at an increased risk of developing a psychotic disorder. Screening has not been well studied in general mental health settings that serve young adults in the age range associated with highest risk for psychosis. This study explored the feasibility of psychosis-risk screening and assessment among help-seeking students at a university counseling center. The PRIME Screen-Revised was administered to students at clinic intake. Participants who screened positively were offered a follow-up assessment using the Structured Interview for Psychosis-risk Syndromes (SIPS). At intake, 510 students completed the PRIME Screen-Revised, with 132 (25.9%) screening positive. Comprehensive psychosis-spectrum evaluations were completed with 38 participants, and 22 met criteria for a psychosis-spectrum disorder, representing 57.9% of this subsample. Findings suggest that psychosis-risk screening in a college clinic is a promising approach to identifying those at high risk for or in the early stages of psychosis.

An analysis of physicians' diagnostic reasoning regarding pediatric abusive head trauma.

BACKGROUND: Physician diagnoses of abusive head trauma (AHT) have been criticized for circular reasoning and over-reliance on a "triad" of findings. Absent a gold standard, analyses that apply restrictive reference standards for AHT and non-AHT could serve to confirm or refute these criticisms. OBJECTIVES: To compare clinical presentations and injuries in patients with witnessed/admitted AHT vs. witnessed non-AHT, and with witnessed/admitted AHT vs. physician diagnosed AHT not witnessed/admitted. To measure the triad's AHT test performance in patients with witnessed/admitted AHT vs. witnessed non-AHT. PARTICIPANTS AND SETTING: Acutely head injured patients <3 years hospitalized for intensive care across 18 sites between 2010 and 2021. METHODS: Secondary analyses of existing, combined, cross-sectional datasets. Probability values and odds ratios were used to identify and characterize differences. Test performance measures included sensitivity, specificity, and predictive values. RESULTS: Compared to patients with witnessed non-AHT (n = 100), patients with witnessed/admitted AHT (n = 58) presented more frequently with respiratory compromise (OR 2.94, 95% CI: 1.50-5.75); prolonged encephalopathy (OR 5.23, 95% CI: 2.51-10.89); torso, ear, or neck bruising (OR 11.87, 95% CI: 4.48-31.48); bilateral subdural hemorrhages (OR 8.21, 95% CI: 3.94-17.13); diffuse brain hypoxia, ischemia, or swelling (OR 6.51, 95% CI: 3.06-13.02); and dense, extensive retinal hemorrhages (OR 7.59, 95% CI: 2.85-20.25). All differences were statistically significant (p ≤ .001). No significant differences were observed in patients with witnessed/admitted AHT (n = 58) vs. patients diagnosed with AHT not witnessed/admitted (n = 438). The triad demonstrated AHT specificity and positive predictive value ≥0.96. CONCLUSIONS: The observed differences in patients with witnessed/admitted AHT vs. witnessed non-AHT substantiate prior reports. The complete absence of differences in patients with witnessed/admitted AHT vs. physician diagnosed AHT not witnessed/admitted supports an impression that physicians apply diagnostic reasoning informed by knowledge of previously reported injury patterns. Concern for abuse is justified in patients who present with "the triad."

Screening for pediatric abusive head trauma: Are three variables enough?

BACKGROUND: The PediBIRN 4-variable clinical decision rule (CDR) detects abusive head trauma (AHT) with 96% sensitivity in pediatric intensive care (PICU) settings. Preliminary analysis of its performance in Pediatric Emergency Department settings found that elimination of its fourth predictor variable enhanced screening accuracy. OBJECTIVE: To compare the AHT screening performances of the "PediBIRN-4" CDR vs. the simplified 3-variable CDR in PICU settings. PARTICIPANTS AND SETTINGS: 973 acutely head-injured children <3 years hospitalized for intensive care across 18 sites between February 2011 and March 2021. METHODS: Retrospective, secondary analysis of the combined, prospective PediBIRN data sets. AHT definitional criteria and physicians' diagnoses were applied iteratively to sort patients into abusive vs. other head trauma cohorts. Outcome measures of CDR performance included sensitivity, specificity, predictive values, likelihood ratios, ROC AUC, and the correlation between each CDR's patient-specific estimates of AHT probability and the overall positive yield of patients' completed abuse evaluations. RESULTS: Applied accurately and consistently, both CDR's would have performed with sensitivity ≥93% and negative predictive value ≥91%. Eliminating the PediBIRN-4's fourth predictor variable resulted in significantly higher specificity (↑'d ≥19%), positive predictive value (↑'d ≥8%), and ROC AUC (↑'d ≥5%), but a 3% reduction in sensitivity. Both CDRs provided patient-specific estimates of abuse probability very strongly correlated with the positive yield of patients' completed abuse evaluations (Pearson's r = 0.95 and 0.91, p = .13). CONCLUSION: The PediBIRN 3-variable CDR performed with greater AHT screening accuracy than the 4-variable CDR. Both are good predictors of the results of patients' subsequent completed abuse evaluations.

Fellow Entrustment for the Common Pediatric Subspecialty Entrustable Professional Activities Across Subspecialties.

OBJECTIVE: To determine the relationship between level of supervision (LOS) ratings for the Common Pediatric Subspecialty Entrustable Professional Activities (EPAs) with their associated subcompetency milestones across subspecialties and by fellowship training year. METHODS: Clinical Competency Committees (CCCs) in 14 pediatric subspecialties submitted LOS ratings for 6 Common Subspecialty EPAs and subcompetency milestone levels mapped to these EPAs. We examined associations between these subcompetency milestone levels and LOS ratings across subspecialty training year by fitting per-EPA linear mixed effects models, regressing LOS rating on milestone level and on training year. RESULTS: CCCs from 211 pediatric fellowship programs provided data for 369 first, 336 second, and 331 third year fellows. Mean subcompetency milestone levels increased similarly among subspecialties for most EPAs compared with the reference, Adolescent Medicine. Mean subcompetency milestones mapped to each EPA and mean EPA LOS ratings generally increased by training year across all subspecialties. CONCLUSIONS: Subcompetency milestones levels mapped to each Common Subspecialty EPA and the EPA LOS ratings increase similarly across subspecialties and by training year, providing validity evidence for using EPA LOS to assess pediatric subspecialty trainee performance. This study supports the development of tools to facilitated the CCC evaluation process across all pediatric subspecialties.

Achieving Entrustable Professional Activities During Fellowship.

BACKGROUND AND OBJECTIVES: Entrustable Professional Activities (EPAs) were developed to assess pediatric fellows. We previously showed that fellowship program directors (FPDs) may graduate fellows who still require supervision. How this compares with their expectations for entrustment of practicing subspecialists is unknown. METHODS: We surveyed US FPDs in 14 pediatric subspecialties through the Subspecialty Pediatrics Investigator Network between April and August 2017. For each of 7 common pediatric subspecialty EPAs, we compared the minimum level of supervision that FPDs required for graduation with the level they expected of subspecialists for safe and effective practice using the Friedman rank sum test and paired t test. We compared differences between subspecialties using linear regression. RESULTS: We collected data from 660 FPDs (response rate 82%). For all EPAs, FPDs did not require fellows to reach the level of entrustment for graduation that they expected of subspecialists to practice (P < .001). FPDs expected the least amount of supervision for the EPAs consultation and handovers. Mean differences between supervision levels for graduation and practice were smaller for clinical EPAs (consultation, handovers, lead a team) when compared with nonclinical EPAs (quality improvement, management, lead the profession and scholarship; P = .001) and were similar across nearly all subspecialties. CONCLUSIONS: Fellowship graduates may need continued development of clinical and nonclinical skills in their early practice period, underscoring a need for continued assessment and mentoring. Graduation readiness must be based on clear requirements, with alignment of FPD expectations and regulatory standards, to ensure quality care for patients.

Minimum Supervision Levels Required by Program Directors for Pediatric Pulmonary Fellow Graduation.

Background: Entrustable professional activities (EPAs) define the essential tasks expected of subspecialists in unsupervised practice. Although EPAs have been piloted in some programs, their use for summative assessment of pediatric pulmonology fellows for graduation has not been studied. Objective: To determine the minimum level of supervision that pediatric pulmonary program directors (PDs) require of their fellows for graduation and compare it with the minimum level of supervision they expect for a practicing subspecialist for the five pediatric pulmonology EPAs. Methods: Using a modified Delphi approach, we developed supervision scales for the five pediatric pulmonology EPAs and conducted a national survey of pediatric pulmonary PDs in the United States through the Subspecialty Pediatric Investigators Network between April 2017 and August 2017. Results: Forty-six pediatric pulmonary PDs completed the survey, representing a response rate of 85%. The majority did not require fellows to be trusted to practice without supervision for graduation for any of the five EPAs (level 5); the median minimum level of supervision they required was 4, equating to indirect supervision for complex cases. The minimum level for graduation, defined by consensus as the level of supervision for which no more than 20% of PDs would want the level to be lower to allow a fellow to graduate, was 3, which corresponded to requiring supervision for both simple and complex cases. There was a statistically significant difference between the minimum level of supervision deemed necessary by PDs for graduation and for practice as a subspecialist for each of the EPAs. Conclusion: Most pediatric pulmonary PDs reported that they would graduate fellows who may still require indirect supervision for the five pediatric pulmonology EPAs. The findings suggest a need for stakeholders to reevaluate the structure and outcomes of training programs and ensure support for pediatric pulmonologists in their early practice period.

Continued Supervision for the Common Pediatric Subspecialty Entrustable Professional Activities May Be Needed Following Fellowship Graduation.

PURPOSE: Entrustable professional activities (EPAs) are one approach to competency-based medical education (CBME), and 7 EPAs have been developed that address content relevant for all pediatric subspecialties. However, it is not known what level of supervision fellowship program directors (FPDs) deem necessary for graduation. The Subspecialty Pediatrics Investigator Network (SPIN) investigated FPD perceptions of the minimum level of supervision required for a trainee to successfully graduate. METHOD: In 2017, SPIN surveyed all FPDs of accredited fellowships for 14 subspecialties. For each EPA, the minimum supervision level for graduation (ranging from observation only to unsupervised practice) was set such that no more than 20% of FPDs would accept a lower level. RESULTS: The survey response rate was 82% (660/802). The minimum supervision level for graduation varied across the 7 EPAs from 2 (direct) to 4 (indirect for complex cases), with significant differences between EPAs. The percentage of FPDs desiring a lower minimum supervision level ranged from 3% to 17%. Compared with the 4 nonclinical EPAs (quality improvement, management, lead within the profession, scholarship), the 3 clinical EPAs (consultation, handover, lead a team) had higher minimum supervision graduation levels (P < .001), with less likelihood that an FPD would graduate a learner below their minimum level (P < .001). CONCLUSIONS: Consensus among FPDs across all pediatric subspecialties demonstrates the potential need for ongoing supervision for graduates in all 7 common pediatric subspecialty EPAs after fellowship. As CBME programs are implemented, processes and infrastructure to support new graduates are important considerations for leaders.

Learner Levels of Supervision Across the Continuum of Pediatrics Training.

PURPOSE: To describe trajectories in level of supervision ratings for linked entrustable professional activities (EPAs) among pediatric learners in medical school, residency, fellowship. METHOD: The authors performed secondary analyses of 3 linked datasets of level of supervision ratings for the Core EPAs for Entering Residency, the General Pediatrics EPAs, and the Subspecialty Pediatrics EPAs. After identifying 9 activities in common across training stages and aligning the level of entrustment-supervision scales across the datasets, piecewise ordinal and linear mixed effects models were fitted to characterize trajectories of supervision ratings. RESULTS: Within each training period, learners were rated as needing less supervision over time in each activity. When transitioning from medical school to residency or during the first year of residency, learners were rated as needing greater supervision in activities related to patient management, teamwork, emergent care, and public health/QI than in earlier periods. When transitioning from residency to fellowship, learners were always rated as needing greater supervision than they had been accorded at the end of residency and sometimes even more than they had been accorded at the start of residency. CONCLUSIONS: Although development over training is often imagined as continuous and monotonically increasing competence, this study provides empirical evidence supporting the idea that entrustment is a set of discrete decisions. The relaxation of supervision in training is not a linear process. Even with a seamless curriculum, supervision is tightly bound to the training setting. Several explanations for these findings are discussed.