Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Validation of the PediBIRN-7 clinical prediction rule for pediatric abusive head trauma.

BACKGROUND: The PediBIRN-7 clinical prediction rule incorporates the (positive or negative) predictive contributions of completed abuse evaluations to estimate abusive head trauma (AHT) probability after abuse evaluation. Applying definitional criteria as proxies for AHT and non-AHT ground truth, it performed with sensitivity 0.73 (95 % CI: 0.66-0.79), specificity 0.87 (95 % CI: 0.82-0.90), and ROC-AUC 0.88 (95 % CI: 0.85-0.92) in its derivation study. OBJECTIVE: To validate the PediBIRN-7's AHT prediction performance in a novel, equivalent, patient population. PARTICIPANTS AND SETTINGS: Consecutive, acutely head-injured children <3 years hospitalized for intensive care across eight sites between 2017 and 2020 with completed skeletal surveys and retinal exams (N = 342). METHODS: Secondary analysis of an existing, cross-sectional, prospective dataset, including assignment of patient-specific estimates of AHT probability, calculation of AHT prediction performance measures (ROC-AUC, sensitivity, specificity, predictive values), and completion of sensitivity analyses to estimate best- and worst-case prediction performances. RESULTS: Applying the same definitional criteria, the PediBIRN-7 performed with sensitivity 0.74 (95 % CI: 0.66-0.81), specificity 0.77 (95 % CI: 0.70-0.83), and ROC-AUC 0.83 (95 % CI: 0.78-0.88). The reduction in ROC-AUC was statistically insignificant (p = .07). Applying physicians' final consensus diagnoses as proxies for AHT and non-AHT ground truth, the PediBIRN-7 performed with sensitivity 0.73 (95 % CI: 0.66-0.79), specificity 0.87 (95 % CI: 0.82-0.90), and ROC-AUC 0.90 (95 % CI: 0.87-0.94). Sensitivity analyses demonstrated minimal changes in rule performance. CONCLUSION: The PediBIRN-7's overall AHT prediction performance has been validated in a novel, equivalent, patient population. Its patient-specific estimates of AHT probability can inform physicians' AHT-related diagnostic reasoning after abuse evaluation.

Epidemiology of brief resolved unexplained events and impact of clinical practice guidelines in general and pediatric emergency departments.

OBJECTIVES: The aim of this study was to describe the incidence of brief resolved unexplained events (BRUEs) and compare the impact of a national clinical practice guideline (CPG) on admission and diagnostic testing practices between general and pediatric emergency departments (EDs). METHODS: Using the Nationwide Emergency Department Sample for 2012-2019, we conducted a cross-sectional study of children <1 year of age with an International Classification of Diseases diagnostic code for BRUE. Population incidence rate was estimated using Centers for Disease Control and Prevention birth data. ED incidence rate was estimated for all ED encounters. We used interrupted time series to evaluate the associated impact of the CPG publication on the outcomes of ED disposition (discharge, admission, and transfer) and electrocardiogram (ECG) use. RESULTS: Of 133,972 encounters for BRUE, 80.0% occurred in general EDs. BRUE population incidence was 4.28 per 1000 live births and the annual incidence remained stable (p = 0.19). BRUE ED incidence was 5.06 per 1000 infant ED encounters (p = 0.14). The impact of the BRUE CPG on admission rates was limited to pediatric EDs (level shift -23.3%, p = 0.002). Transfers from general EDs did not change with the CPG (level shift 2.2%, p = 0.17). After the CPG was published, ECGs increased by 13.7% in pediatric EDs (p = 0.005) but did not change in general EDs (level shift -0.2%, p = 0.82). CONCLUSIONS: BRUEs remain a common pediatric problem at a population level and in EDs. Although a disproportionate number of infants present to general EDs, there is differential uptake of the CPG recommendations between pediatric and general EDs. These findings may support quality improvement opportunities aimed at improving care for these infants and decreasing unnecessary hospital admissions or transfers.

Exploring Factors for Implementation of EPAs in Pediatric Subspecialty Fellowships: A Qualitative Study of Program Directors.

OBJECTIVE: To understand fellowship program directors' (FPDs) perspectives on facilitators and barriers to using entrustable professional activities (EPAs) in pediatric subspecialty training. METHODS: We performed a qualitative study of FPDs, balancing subspecialty, program size, geographic region and current uses of EPAs. A study coordinator conducted 1-on-1 interviews using a semistructured approach to explore EPA use or nonuse and factors supporting or preventing their use. Investigators independently coded transcribed interviews using an inductive approach and the constant comparative method. Group discussion informed code structure development and refinement. Iterative data collection and analysis continued until theoretical sufficiency was achieved, yielding a thematic analysis. RESULTS: Twenty-eight FPDs representing 11 pediatric subspecialties were interviewed, of whom 16 (57%) reported current EPA use. Five major themes emerged: (1) facilitators including the intuitive nature and simple wording of EPAs; (2) barriers such as workload burden and lack of a regulatory requirement; (2) variable knowledge and training surrounding EPAs, leading to differing levels of understanding; (3) limited current use of EPAs, even among self-reported users; and (4) complementary nature of EPAs and milestones. FPDs acknowledged the differing strengths of both EPAs and milestones but sought additional knowledge about the value added by EPAs for assessing trainees, including the impact on outcomes. CONCLUSIONS: Identified themes can inform effective and meaningful EPA implementation strategies: Supporting and educating FPDs, ongoing assessment of the value of EPAs in training, and practical integration with current workflow. Generating additional data and engaging stakeholders is critical for successful implementation for the pediatric subspecialties.

Retinal hemorrhage variation in inertial versus contact head injuries.

BACKGROUND: Abusive head trauma (AHT) is frequently accompanied by dense/extensive retinal hemorrhages to the periphery with or without retinoschisis (complex retinal hemorrhages, cRH). cRH are uncommon without AHT or major trauma. OBJECTIVE: The study objectives were to determine whether cRH are associated with inertial vs. contact mechanisms and are primary vs. secondary injuries. PARTICIPANTS AND SETTING: This retrospective study utilized a de-identified PediBIRN database of 701 children <3-years-old presenting to intensive care for head trauma. Children with motor vehicle related trauma and preexisting brain abnormalities were excluded. All had imaging showing head injury and a dedicated ophthalmology examination. METHODS: Contact injuries included craniofacial soft tissue injuries, skull fractures and epidural hematoma. Inertial injuries included acute impairment or loss of consciousness and/or bilateral and/or interhemispheric subdural hemorrhage. Abuse was defined in two ways, by 1) predetermined criteria and 2) caretaking physicians/multidisciplinary team's diagnostic consensus. RESULTS: PediBIRN subjects with cRH frequently experienced inertial injury (99.4 % (308/310, OR = 53.74 (16.91-170.77)) but infrequently isolated contact trauma (0.6 % (2/310), OR = 0.02 (0.0004-0.06)). Inertial injuries predominated over contact trauma among children with cRH sorted AHT by predetermined criteria (99.1 % (237/239), OR = 20.20 (6.09-67.01) vs 0.5 % (2/339), OR = 0.04 (0.01-0.17)). Fifty-nine percent of patients with cRH, <24 h altered consciousness, and inertial injuries lacked imaging evidence of brain hypoxia, ischemia, or swelling. CONCLUSIONS: cRH are significantly associated with inertial angular acceleration forces. They can occur without brain hypoxia, ischemia or swelling suggesting they are not secondary injuries.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment.

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Skeletal survey yields in low vs. high risk pediatric patients with skull fractures.

BACKGROUND: To assess for occult fractures, physicians often opt to obtain skeletal surveys (SS) in young, acutely head-injured patients who present with skull fractures. Data informing optimal decision management are lacking. OBJECTIVE: To determine the positive yields of radiologic SS in young patients with skull fractures presumed to be at low vs. high risk for abuse. PARTICIPANTS AND SETTING: 476 acutely head injured, skull-fractured patients <3 years hospitalized for intensive care across 18 sites between February 2011 and March 2021. METHODS: We conducted a retrospective, secondary analysis of the combined, prospective Pediatric Brain Injury Research Network (PediBIRN) data set. RESULTS: 204 (43 %) of 476 patients had simple, linear, parietal skull fractures. 272 (57 %) had more complex skull fracture(s). Only 315 (66 %) of 476 patients underwent SS, including 102 (32 %) patients presumed to be at low risk for abuse (patients who presented with a consistent history of accidental trauma; intracranial injuries no deeper than the cortical brain; and no respiratory compromise, alteration or loss of consciousness, seizures, or skin injuries suspicious for abuse). Only one of 102 low risk patients revealed findings indicative of abuse. In two other low risk patients, SS helped to confirm metabolic bone disease. CONCLUSIONS: Less than 1 % of low risk patients under three years of age who presented with simple or complex skull fracture(s) revealed other abusive fractures. Our results could inform efforts to reduce unnecessary skeletal surveys.

Clinical competency committee perceptions of entrustable professional activities and their value in assessing fellows: A qualitative study of pediatric subspecialty program directors.

OBJECTIVES: To examine the composition and processes of Clinical Competency Committees (CCCs) assigning entrustable professional activity (EPA) levels of supervision for pediatric subspecialty fellows and to examine fellowship program director (FPD) perspectives about using EPAs to determine fellows' graduation readiness. METHODS: A qualitative study was performed using one-on-one interviews with a purposeful sample of pediatric subspecialty FPDs to yield a thematic analysis. Semi-structured interview guides were used for participants who self-identified as EPA users or non-users. Inductive analysis and coding were performed on transcripts until theoretical sufficiency was attained. RESULTS: Twenty-eight FPDs were interviewed. There was significant variability in the composition and processes of CCCs across subspecialties. FPDs felt that CCCs intuitively understand what entrustment means, allowing for ease of application of level of supervision (LOS) scales and consensus. FPDs perceived that EPAs provided a global assessment of fellows and are one tool to determine graduation readiness. CONCLUSIONS: Although there was variability in the makeup and processes of CCCs across subspecialties, FPDs believe EPAs are intuitive and relatively easy to implement. Consensus can be reached easily using EPA-specific LOS scales focusing on entrustment. FPDs desire a better understanding of how EPAs should be used for graduation.

Diagnostic testing for and detection of physical abuse in infants with brief resolved unexplained events.

BACKGROUND: A Brief Resolved Unexplained Event (BRUE) can be a sign of occult physical abuse. OBJECTIVES: To identify rates of diagnostic testing able to detect physical abuse (head imaging, skeletal survey, and liver transaminases) at BRUE presentation. The secondary objective was to estimate the rate of physical abuse diagnosed at initial BRUE presentation through 1 year of age. PARTICIPANTS AND SETTING: Infants who presented with a BRUE at one of 15 academic or community hospitals were followed from initial BRUE presentation until 1 year of age for BRUE recurrence or revisits. METHODS: This study was part of the BRUE Research and Quality Improvement Network, a multicenter retrospective cohort examining infants with BRUE. Generalized estimating equations assessed associations with performance of diagnostic testing (adjusted odds ratio (aOR)). RESULTS: Of the 2036 infants presenting with a BRUE, 6.2 % underwent head imaging, 7.0 % skeletal survey, and 12.1 % liver transaminases. Infants were more likely to undergo skeletal survey if there were physical examination findings concerning for trauma (aOR 8.23, 95 % CI [1.92, 35.24], p < 0.005) or concerning social history (aOR 1.89, 95 % CI [1.13, 3.16], p = 0.015). There were 7 (0.3 %) infants diagnosed with physical abuse: one at BRUE presentation, one <3 days after BRUE presentation, and five >30 days after BRUE presentation. CONCLUSION: There were low rates of diagnostic testing and physical abuse identified in infants presenting with BRUE. Further study including standardized testing protocols is warranted to identify physical abuse in infants presenting with a BRUE.

Pediatric Emergency Medicine Fellowship Program Directors' Viewpoint: Minimum Levels of Entrustment for Graduating Fellows and Practicing Physicians to Perform the Subspecialty's Professional Activities.

OBJECTIVES: Entrustable Professional Activities (EPAs) are essential tasks physicians perform within their professions. Entrustment levels that pediatric emergency medicine (PEM) fellowship program directors (FPDs) expect graduating fellows to achieve for PEM-specific and common pediatric subspecialty EPAs remain unreported. This study aims to determine minimum entrustment levels FPDs require fellows to achieve to graduate from fellowship and to compare FPD expectations for fellows versus practicing PEM physicians. METHODS: Secondary analysis of PEM-specific data from a national multispecialty cross-sectional survey of pediatric subspecialty FPDs. For 6 PEM-specific and 7 common pediatric subspecialty EPAs, PEM FPDs indicated (1) minimum entrustment levels fellows should achieve by training completion, (2) whether they would allow a fellow to graduate below these minimum levels, and (3) minimum levels for safe and effective practice by PEM physicians. Minimum levels were defined as the level that more than 80% of FPDs would not drop below. RESULTS: Sixty of 77 PEM FPDs (78%) completed the survey. Most respondents did not require fellows to achieve the highest level (level 5-no supervision) by graduation for any PEM-specific EPAs. The median level FPDs expected for practicing PEM physicians was 5 (trusted to perform without supervision) for EPAs 1 and 4 and level 4 (indirect supervision for complex cases) for the remaining PEM-specific EPAs. Minimum levels expected by FPDs for common subspecialty pediatric EPAs were lower for both groups. CONCLUSIONS: Most PEM FPDs indicated that they would graduate fellows before their achievement of the highest entrustment level for all EPAs. Most also indicated that they do not expect practicing PEM physicians to perform all EPAs without supervision. These findings indicate need for stakeholders to evaluate current structure and outcomes of PEM fellowship programs and for institutions and organizations to ensure adequate support in time and resources for ongoing learning for practicing PEM physicians.

The association of subcortical brain injury and abusive head trauma.

BACKGROUND: Abusive head trauma (AHT) remains a major pediatric problem with diagnostic challenges. A small pilot study previously associated subcortical brain injury with AHT. OBJECTIVES: To investigate the association of subcortical injury on neuroimaging with the diagnosis of AHT. PARTICIPANTS AND SETTING: Children <3 years with acute TBI admitted to 18 PICUs between 2011 and 2021. METHODS: Secondary analysis of existing, combined, de-identified, cross-sectional dataset. RESULTS: Deepest location of visible injury was characterized as scalp/skull/epidural (n = 170), subarachnoid/subdural (n = 386), cortical brain (n = 170), or subcortical brain (n = 247) (total n = 973). Subcortical injury was significantly associated with AHT using both physicians' diagnostic impression (OR: 8.41 [95 % CI: 5.82-12.44]) and a priori definitional criteria (OR: 5.99 [95 % CI: 4.31-8.43]). Caregiver reports consistent with the child's gross motor skills and historically consistent with repetition decreased as deepest location of injury increased, p < 0.001. Patients with subcortical injuries were significantly more likely to have traumatic extracranial injuries such as rib fractures (OR 3.36, 95 % CI 2.30-4.92) or retinal hemorrhages (OR 5.97, 95 % CI 4.35-8.24), respiratory compromise (OR 12.12, 95 % CI 8.49-17.62), circulatory compromise (OR 6.71, 95 % CI 4.87-9.29), seizures (OR 3.18, 95 % CI 2.35-4.29), and acute encephalopathy (OR 12.44, 95 % CI 8.16-19.68). CONCLUSIONS: Subcortical injury is associated with a diagnosis of AHT, historical inaccuracies concerning for abuse, traumatic extracranial injuries, and increased severity of illness including respiratory and circulatory compromise, seizures, and prolonged loss of consciousness. Presence of subcortical injury should be considered as one component of the complex AHT diagnostic process.

An analysis of physicians' diagnostic reasoning regarding pediatric abusive head trauma.

BACKGROUND: Physician diagnoses of abusive head trauma (AHT) have been criticized for circular reasoning and over-reliance on a "triad" of findings. Absent a gold standard, analyses that apply restrictive reference standards for AHT and non-AHT could serve to confirm or refute these criticisms. OBJECTIVES: To compare clinical presentations and injuries in patients with witnessed/admitted AHT vs. witnessed non-AHT, and with witnessed/admitted AHT vs. physician diagnosed AHT not witnessed/admitted. To measure the triad's AHT test performance in patients with witnessed/admitted AHT vs. witnessed non-AHT. PARTICIPANTS AND SETTING: Acutely head injured patients <3 years hospitalized for intensive care across 18 sites between 2010 and 2021. METHODS: Secondary analyses of existing, combined, cross-sectional datasets. Probability values and odds ratios were used to identify and characterize differences. Test performance measures included sensitivity, specificity, and predictive values. RESULTS: Compared to patients with witnessed non-AHT (n = 100), patients with witnessed/admitted AHT (n = 58) presented more frequently with respiratory compromise (OR 2.94, 95% CI: 1.50-5.75); prolonged encephalopathy (OR 5.23, 95% CI: 2.51-10.89); torso, ear, or neck bruising (OR 11.87, 95% CI: 4.48-31.48); bilateral subdural hemorrhages (OR 8.21, 95% CI: 3.94-17.13); diffuse brain hypoxia, ischemia, or swelling (OR 6.51, 95% CI: 3.06-13.02); and dense, extensive retinal hemorrhages (OR 7.59, 95% CI: 2.85-20.25). All differences were statistically significant (p ≤ .001). No significant differences were observed in patients with witnessed/admitted AHT (n = 58) vs. patients diagnosed with AHT not witnessed/admitted (n = 438). The triad demonstrated AHT specificity and positive predictive value ≥0.96. CONCLUSIONS: The observed differences in patients with witnessed/admitted AHT vs. witnessed non-AHT substantiate prior reports. The complete absence of differences in patients with witnessed/admitted AHT vs. physician diagnosed AHT not witnessed/admitted supports an impression that physicians apply diagnostic reasoning informed by knowledge of previously reported injury patterns. Concern for abuse is justified in patients who present with "the triad."

Screening for pediatric abusive head trauma: Are three variables enough?

BACKGROUND: The PediBIRN 4-variable clinical decision rule (CDR) detects abusive head trauma (AHT) with 96% sensitivity in pediatric intensive care (PICU) settings. Preliminary analysis of its performance in Pediatric Emergency Department settings found that elimination of its fourth predictor variable enhanced screening accuracy. OBJECTIVE: To compare the AHT screening performances of the "PediBIRN-4" CDR vs. the simplified 3-variable CDR in PICU settings. PARTICIPANTS AND SETTINGS: 973 acutely head-injured children <3 years hospitalized for intensive care across 18 sites between February 2011 and March 2021. METHODS: Retrospective, secondary analysis of the combined, prospective PediBIRN data sets. AHT definitional criteria and physicians' diagnoses were applied iteratively to sort patients into abusive vs. other head trauma cohorts. Outcome measures of CDR performance included sensitivity, specificity, predictive values, likelihood ratios, ROC AUC, and the correlation between each CDR's patient-specific estimates of AHT probability and the overall positive yield of patients' completed abuse evaluations. RESULTS: Applied accurately and consistently, both CDR's would have performed with sensitivity ≥93% and negative predictive value ≥91%. Eliminating the PediBIRN-4's fourth predictor variable resulted in significantly higher specificity (↑'d ≥19%), positive predictive value (↑'d ≥8%), and ROC AUC (↑'d ≥5%), but a 3% reduction in sensitivity. Both CDRs provided patient-specific estimates of abuse probability very strongly correlated with the positive yield of patients' completed abuse evaluations (Pearson's r = 0.95 and 0.91, p = .13). CONCLUSION: The PediBIRN 3-variable CDR performed with greater AHT screening accuracy than the 4-variable CDR. Both are good predictors of the results of patients' subsequent completed abuse evaluations.

Fellow Entrustment for the Common Pediatric Subspecialty Entrustable Professional Activities Across Subspecialties.

OBJECTIVE: To determine the relationship between level of supervision (LOS) ratings for the Common Pediatric Subspecialty Entrustable Professional Activities (EPAs) with their associated subcompetency milestones across subspecialties and by fellowship training year. METHODS: Clinical Competency Committees (CCCs) in 14 pediatric subspecialties submitted LOS ratings for 6 Common Subspecialty EPAs and subcompetency milestone levels mapped to these EPAs. We examined associations between these subcompetency milestone levels and LOS ratings across subspecialty training year by fitting per-EPA linear mixed effects models, regressing LOS rating on milestone level and on training year. RESULTS: CCCs from 211 pediatric fellowship programs provided data for 369 first, 336 second, and 331 third year fellows. Mean subcompetency milestone levels increased similarly among subspecialties for most EPAs compared with the reference, Adolescent Medicine. Mean subcompetency milestones mapped to each EPA and mean EPA LOS ratings generally increased by training year across all subspecialties. CONCLUSIONS: Subcompetency milestones levels mapped to each Common Subspecialty EPA and the EPA LOS ratings increase similarly across subspecialties and by training year, providing validity evidence for using EPA LOS to assess pediatric subspecialty trainee performance. This study supports the development of tools to facilitated the CCC evaluation process across all pediatric subspecialties.

Achieving Entrustable Professional Activities During Fellowship.

BACKGROUND AND OBJECTIVES: Entrustable Professional Activities (EPAs) were developed to assess pediatric fellows. We previously showed that fellowship program directors (FPDs) may graduate fellows who still require supervision. How this compares with their expectations for entrustment of practicing subspecialists is unknown. METHODS: We surveyed US FPDs in 14 pediatric subspecialties through the Subspecialty Pediatrics Investigator Network between April and August 2017. For each of 7 common pediatric subspecialty EPAs, we compared the minimum level of supervision that FPDs required for graduation with the level they expected of subspecialists for safe and effective practice using the Friedman rank sum test and paired t test. We compared differences between subspecialties using linear regression. RESULTS: We collected data from 660 FPDs (response rate 82%). For all EPAs, FPDs did not require fellows to reach the level of entrustment for graduation that they expected of subspecialists to practice (P < .001). FPDs expected the least amount of supervision for the EPAs consultation and handovers. Mean differences between supervision levels for graduation and practice were smaller for clinical EPAs (consultation, handovers, lead a team) when compared with nonclinical EPAs (quality improvement, management, lead the profession and scholarship; P = .001) and were similar across nearly all subspecialties. CONCLUSIONS: Fellowship graduates may need continued development of clinical and nonclinical skills in their early practice period, underscoring a need for continued assessment and mentoring. Graduation readiness must be based on clear requirements, with alignment of FPD expectations and regulatory standards, to ensure quality care for patients.

Minimum Supervision Levels Required by Program Directors for Pediatric Pulmonary Fellow Graduation.

Background: Entrustable professional activities (EPAs) define the essential tasks expected of subspecialists in unsupervised practice. Although EPAs have been piloted in some programs, their use for summative assessment of pediatric pulmonology fellows for graduation has not been studied. Objective: To determine the minimum level of supervision that pediatric pulmonary program directors (PDs) require of their fellows for graduation and compare it with the minimum level of supervision they expect for a practicing subspecialist for the five pediatric pulmonology EPAs. Methods: Using a modified Delphi approach, we developed supervision scales for the five pediatric pulmonology EPAs and conducted a national survey of pediatric pulmonary PDs in the United States through the Subspecialty Pediatric Investigators Network between April 2017 and August 2017. Results: Forty-six pediatric pulmonary PDs completed the survey, representing a response rate of 85%. The majority did not require fellows to be trusted to practice without supervision for graduation for any of the five EPAs (level 5); the median minimum level of supervision they required was 4, equating to indirect supervision for complex cases. The minimum level for graduation, defined by consensus as the level of supervision for which no more than 20% of PDs would want the level to be lower to allow a fellow to graduate, was 3, which corresponded to requiring supervision for both simple and complex cases. There was a statistically significant difference between the minimum level of supervision deemed necessary by PDs for graduation and for practice as a subspecialist for each of the EPAs. Conclusion: Most pediatric pulmonary PDs reported that they would graduate fellows who may still require indirect supervision for the five pediatric pulmonology EPAs. The findings suggest a need for stakeholders to reevaluate the structure and outcomes of training programs and ensure support for pediatric pulmonologists in their early practice period.

A Cluster Randomized Trial to Reduce Missed Abusive Head Trauma in Pediatric Intensive Care Settings.

OBJECTIVE: To estimate the impact of the PediBIRN (Pediatric Brain Injury Research Network) 4-variable clinical decision rule (CDR) on abuse evaluations and missed abusive head trauma in pediatric intensive care settings. STUDY DESIGN: This was a cluster randomized trial. Participants included 8 pediatric intensive care units (PICUs) in US academic medical centers; PICU and child abuse physicians; and consecutive patients with acute head injures <3 years (n = 183 and n = 237, intervention vs control). PICUs were stratified by patient volumes, pair-matched, and randomized equally to intervention or control conditions. Randomization was concealed from the biostatistician. Physician-directed, cluster-level interventions included initial and booster training, access to an abusive head trauma probability calculator, and information sessions. Outcomes included "higher risk" patients evaluated thoroughly for abuse (with skeletal survey and retinal examination), potential cases of missed abusive head trauma (patients lacking either evaluation), and estimates of missed abusive head trauma (among potential cases). Group comparisons were performed using generalized linear mixed-effects models. RESULTS: Intervention physicians evaluated a greater proportion of higher risk patients thoroughly (81% vs 73%, P = .11) and had fewer potential cases of missed abusive head trauma (21% vs 32%, P = .05), although estimated cases of missed abusive head trauma did not differ (7% vs 13%, P = .22). From baseline (in previous studies) to trial, the change in higher risk patients evaluated thoroughly (67%→81% vs 78%→73%, P = .01), and potential cases of missed abusive head trauma (40%→21% vs 29%→32%, P = .003), diverged significantly. We did not identify a significant divergence in the number of estimated cases of missed abusive head trauma (15%→7% vs 11%→13%, P = .22). CONCLUSIONS: PediBIRN-4 CDR application facilitated changes in abuse evaluations that reduced potential cases of missed abusive head trauma in PICU settings. TRIAL REGISTRATION: ClinicalTrials.gov: NCT03162354.

Validation of a Clinical Decision Rule to Predict Abuse in Young Children Based on Bruising Characteristics.

Importance: Bruising caused by physical abuse is the most common antecedent injury to be overlooked or misdiagnosed as nonabusive before an abuse-related fatality or near-fatality in a young child. Bruising occurs from both nonabuse and abuse, but differences identified by a clinical decision rule may allow improved and earlier recognition of the abused child. Objective: To refine and validate a previously derived bruising clinical decision rule (BCDR), the TEN-4 (bruising to torso, ear, or neck or any bruising on an infant <4.99 months of age), for identifying children at risk of having been physically abused. Design, Setting, and Participants: This prospective cross-sectional study was conducted from December 1, 2011, to March 31, 2016, at emergency departments of 5 urban children's hospitals. Children younger than 4 years with bruising were identified through deliberate examination. Statistical analysis was completed in June 2020. Exposures: Bruising characteristics in 34 discrete body regions, patterned bruising, cumulative bruise counts, and patient's age. The BCDR was refined and validated based on these variables using binary recursive partitioning analysis. Main Outcomes and Measures: Injury from abusive vs nonabusive trauma was determined by the consensus judgment of a multidisciplinary expert panel. Results: A total of 21 123 children were consecutively screened for bruising, and 2161 patients (mean [SD] age, 2.1 [1.1] years; 1296 [60%] male; 1785 [83%] White; 1484 [69%] non-Hispanic/Latino) were enrolled. The expert panel achieved consensus on 2123 patients (98%), classifying 410 (19%) as abuse and 1713 (79%) as nonabuse. A classification tree was fit to refine the rule and validated via bootstrap resampling. The resulting BCDR was 95.6% (95% CI, 93.0%-97.3%) sensitive and 87.1% (95% CI, 85.4%-88.6%) specific for distinguishing abuse from nonabusive trauma based on body region bruised (torso, ear, neck, frenulum, angle of jaw, cheeks [fleshy], eyelids, and subconjunctivae), bruising anywhere on an infant 4.99 months and younger, or patterned bruising (TEN-4-FACESp). Conclusions and Relevance: In this study, an affirmative finding for any of the 3 BCDR TEN-4-FACESp components in children younger than 4 years indicated a potential risk for abuse; these results warrant further evaluation. Clinical application of this tool has the potential to improve recognition of abuse in young children with bruising.

Agreement of Program Directors With Clinical Competency Committees for Fellow Entrustment.

OBJECTIVES: Fellowship program directors (FPD) and Clinical Competency Committees (CCCs) both assess fellow performance. We examined the association of entrustment levels determined by the FPD with those of the CCC for 6 common pediatric subspecialty entrustable professional activities (EPAs), hypothesizing there would be strong correlation and minimal bias between these raters. METHODS: The FPDs and CCCs separately assigned a level of supervision to each of their fellows for 6 common pediatric subspecialty EPAs. For each EPA, we determined the correlation between FPD and CCC assessments and calculated bias as CCC minus FPD values for when the FPD was or was not a member of the CCC. In addition, we examined the effect of program size, FPD understanding of EPAs, and subspecialty on the correlations. Data were obtained in fall 2014 and spring 2015. RESULTS: A total of 1040 fellows were assessed in the fall and 1048 in the spring. In both periods and for each EPA, there was a strong correlation between FPD and CCC supervision levels (P < .001). The correlation was somewhat lower when the FPD was not a CCC member (P < .001). Overall bias in both periods was small. CONCLUSIONS: The correlation between FPD and CCC assignment of EPA supervision levels is strong. Although slightly weaker when the FPD is not a CCC member, bias is small, so this is likely unimportant in determining fellow entrustment level. The similar performance ratings of FPDs and CCCs support the validity argument for EPAs as competency-based assessment tools.

Shared Mental Models Among Clinical Competency Committees in the Context of Time-Variable, Competency-Based Advancement to Residency.

PURPOSE: To evaluate response process validity evidence for clinical competency committee (CCC) assessments of first-year residents on a subset of General Pediatrics Entrustable Professional Activities (EPAs) and milestones in the context of a national pilot of competency-based, time-variable (CBTV) advancement from undergraduate to graduate medical education. METHOD: Assessments of 2 EPAs and 8 milestones made by the trainees' actual CCCs and 2 different blinded "virtual" CCCs for 48 first-year pediatrics residents at 4 residency programs between 2016 and 2018 were compared. Residents had 3 different training paths from medical school to residency: time-variable graduation at the same institution as their residency, time-fixed graduation at the same institution, or time-fixed graduation from a different institution. Assessments were compared using ordinal mixed-effects models. RESULTS: Actual CCCs assigned residents higher scores than virtual CCCs on milestones and one EPA's supervision levels. Residents who graduated from a different institution than their residency received lower milestone ratings than either group from the same institution; CBTV residents received higher ratings on one milestone (ICS4) and similar ratings on all others compared with non-CBTV residents who completed medical school at the same institution. CONCLUSIONS: First-year residents who graduated from CBTV medical school programs were assessed as having the same level of competence as residents who graduated from traditional medical school programs, but response process evidence suggests that members of CCCs may also draw on undocumented personal knowledge of the learner to draw conclusions about resident competence.