Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment.

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Child Health Needs and the Developmental-Behavioral Pediatrics Workforce Supply: 2020-2040.

Developmental-behavioral pediatrics (DBP) subspecialists care for children with complex neurodevelopmental and behavioral health conditions; additional roles include education and training, advocacy, and research. In 2023, there were 1.0 DBP subspecialists per 100 000 US children aged 0 to 17 years (range 0.0-3.8), with wide variability in DBP subspecialist distribution. Given the prevalence of DB conditions, the current workforce is markedly inadequate to meet the needs of patients and families. The American Board of Pediatrics Foundation led a modeling project to forecast the US pediatric subspecialty workforce from 2020 to 2040 using current trends in each subspecialty. The model predicts workforce supply at baseline and across alternative scenarios and reports results in headcount (HC) and HC adjusted for percent time spent in clinical care, termed "clinical workforce equivalent." For DBP, the baseline model predicts HC growth nationally (+45%, from 669 to 958), but these extremely low numbers translate to minimal patient care impact. Adjusting for population growth over time, projected HC increases from 0.8 to 1.0 and clinical workforce equivalent from 0.5 to 0.6 DBP subspecialists per 100 000 children aged 0 to 18 years by 2040. Even in the best-case scenario (+12.5% in fellows by 2030 and +7% in time in clinical care), the overall numbers would be minimally affected. These current and forecasted trends should be used to shape much-needed solutions in education, training, practice, policy, and workforce research to increase the DBP workforce and improve overall child health.

Level of Supervision for the Entrustable Professional Activities Common to General Pediatrics and the Subspecialties Decreases from Residency to Fellowship.

OBJECTIVES: To compare level of supervision (LOS) ratings of graduating pediatric residents with their assessments as fellows for the five Entrustable Professional Activities (EPAs) common to general pediatrics and the subspecialties and to determine if the difference between ratings from residency to fellowship is less for the QI and Practice Management EPAs, since the skills needed to perform these may be less context-dependent. METHODS: We compared ratings of graduating residents with their assessments as fellows using LOS data from two sequential EPA studies. RESULTS: There were 65 ratings from 41 residents at the first fellow assessment. At graduation, most residents needed little to no supervision for all EPAs with 94% (61/65) of ratings level four or five. In contrast, only 5/65 (8%) of the first fellow assessments were level four or five. The ratings difference for the QI and Practice Management EPAs was similar to the others. CONCLUSIONS: LOS ratings for the EPAs common to generalists and subspecialists reset as residents become fellows. There was no evidence that the QI and Practice Management EPAs are less context-dependent. This study provides additional validity evidence for using these LOS scales to assess trainees in pediatric residency and fellowship.

Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Defining Developmental-Behavioral Pediatrics.

There is an insufficient number of specialty developmental-behavioral pediatrics (DBP) physicians, despite nearly 25% of children and adolescents having a developmental, learning, behavioral, or emotional problem. In the nearly 20 years since becoming a board-certified subspecialty, the definition of DBP clinical practice remains somewhat unclear. This lack of clarity likely contributes to recruitment challenges and workforce issues, and limited visibility of DBP among parents, other professionals, payors, and administrators. Defining DBP is therefore an important step in the survival and growth of the field. In this paper, we describe the methodology used to develop this definition along with the origins of DBP, the persistent challenges to defining its scope, what training in DBP involves, and what distinguishes DBP from other overlapping fields of medicine. We propose the following definition of DBP: developmental-behavioral pediatrics (DBP) is a board-certified, medical subspecialty that cares for children with complex and severe DBP problems by recognizing the multifaceted influences on the development and behavior of children and addressing them through systems-based practice and a neurodevelopmental, strength-based approach that optimizes functioning. Developmental behavioral pediatricians care for children from birth through young adulthood along a continuum including those suspected of, at risk for, or known to have developmental and behavioral disorders.

Clinician Diagnostic Certainty and the Role of the Autism Diagnostic Observation Schedule in Autism Spectrum Disorder Diagnosis in Young Children.

Importance: Autism spectrum disorder (ASD) affects 1 in 44 children. The Autism Diagnostic Observation Schedule (ADOS) is a semi-structured observation developed for use in research but is considered a component of gold standard clinical diagnosis. The ADOS adds time and cost to diagnostic assessments. Objective: To evaluate consistency between clinical diagnosis (index ASD diagnosis) and diagnosis incorporating the ADOS (reference standard ASD diagnosis) and to examine clinician and child factors that predict consistency between index diagnoses and reference standard diagnoses. Design, Setting, and Participants: This prospective diagnostic study was conducted between May 2019 and February 2020. Developmental-behavioral pediatricians (DBPs) made a diagnosis based on clinical assessment (index ASD diagnosis). The ADOS was then administered, after which the DBP made a second diagnosis (reference standard ASD diagnosis). DBPs self-reported diagnostic certainty at the time of the index diagnoses and reference standard diagnoses. The study took place at 8 sites (7 US and 1 European) that provided subspecialty assessments for children with concerns for ASD. Participants included children aged 18 months to 5 years, 11 months, without a prior ASD diagnosis, consecutively referred for possible ASD. Among 648 eligible children, 23 refused, 376 enrolled, and 349 completed the study. All 40 eligible DBPs participated. Exposures: ADOS administered to all child participants. Main Outcomes and Measures: Index diagnoses and reference standard diagnoses of ASD (yes/no). Results: Among the 349 children (279 [79.7%] male; mean [SD] age, 39.9 [13.4] months), index diagnoses and reference standard diagnoses were consistent for 314 (90%) (ASD = 250; not ASD = 64) and changed for 35. Clinician diagnostic certainty was the most sensitive and specific predictor of diagnostic consistency (area under curve = 0.860; P < .001). In a multilevel logistic regression, no child or clinician factors improved prediction of diagnostic consistency based solely on clinician diagnostic certainty at time of index diagnosis. Conclusions and Relevance: In this prospective diagnostic study, clinical diagnoses of ASD by DBPs with vs without the ADOS were consistent in 90.0% of cases. Clinician diagnostic certainty predicted consistency of index diagnoses and reference standard diagnoses. This study suggests that the ADOS is generally not required for diagnosis of ASD in young children by DBPs and that DBPs can identify children for whom the ADOS may be needed.

Pilot Study of Maternal Autoantibody-Related Autism.

OBJECTIVE: The objective of this study was to investigate the presence of maternal autoantibody-related autism spectrum disorder (MAR-ASD) in 2 geographically distinct DBPNet clinical sites (Pennsylvania and Arkansas). MAR-ASD is a biologically defined subtype of ASD that is defined by the presence of autoantibodies specific to proteins in the fetal brain and present in approximately 20% of a Northern California sample but has not been studied in other states. METHODS: Sixty-eight mothers of children with ASD were recruited from 2 DBPNet clinics and provided blood samples. Mothers also completed behavioral questionnaires about their children, and data from the child's clinical diagnostic assessment were abstracted. RESULTS: The mean age of mothers was 38.5 ± 6.1 years, and the mean age of children was 8.3 ± 2.7 years. MAR-ASD was present in 24% of the sample and similar across sites. Children of +MAR mothers had more severe autism symptoms as measured by Autism Diagnostic Observation Schedule comparison scores (W = 3604; p < 0.001) and the Social Communication Questionnaire (W = 4556; p < 0.001). There were no differences in IQ, adaptive function, or aberrant behavior. CONCLUSION: MAR-ASD is a subtype of autism that is present in similar frequencies across 3 states and related to autism severity.

Benefits and Limitations of Virtual Recruitment: Perspectives From Subspeciality Directors.

BACKGROUND AND OBJECTIVES: Because of the coronavirus disease 2019 pandemic and recommendations from a range of leaders and organizations, the pediatrics subspecialty 2020 recruitment season was entirely virtual. Minimal data exist on the effect of this change to guide future strategies. The aim of this study was to understand the effects of virtual recruitment on pediatric subspecialty programs as perceived by program leaders. METHODS: This concurrent, triangulation, mixed-methods study used a survey that was developed through an iterative (3 cycles), consensus-building, modified Delphi process and sent to all pediatric subspecialty program directors (PSPDs) between April and May 2021. Descriptive statistics and thematic analysis were used, and a conceptual framework was developed. RESULTS: Forty-two percent (352 of 840) of PSPDs responded from 16 of the 17 pediatric (94%) subspecialties; 60% felt the virtual interview process was beneficial to their training program. A majority of respondents (72%) reported cost savings were a benefit; additional benefits included greater efficiency of time, more applicants per day, greater faculty involvement, and perceived less time away from residency for applicants. PSPDs reported a more diverse applicant pool. Without an in-person component, PSPDs worried about programs and applicants missing informative, in-person interactions and applicants missing hospital tours and visiting the city. A model based upon theory of change was developed to aid program considerations for future application cycles. CONCLUSIONS: PSPDs identified several benefits to virtual recruitment, including ease of accommodating increased applicants with a diverse applicant pool and enhanced faculty involvement. Identified limitations included reduced interaction between the applicant and the larger institution/city.

Fellow Entrustment for the Common Pediatric Subspecialty Entrustable Professional Activities Across Subspecialties.

OBJECTIVE: To determine the relationship between level of supervision (LOS) ratings for the Common Pediatric Subspecialty Entrustable Professional Activities (EPAs) with their associated subcompetency milestones across subspecialties and by fellowship training year. METHODS: Clinical Competency Committees (CCCs) in 14 pediatric subspecialties submitted LOS ratings for 6 Common Subspecialty EPAs and subcompetency milestone levels mapped to these EPAs. We examined associations between these subcompetency milestone levels and LOS ratings across subspecialty training year by fitting per-EPA linear mixed effects models, regressing LOS rating on milestone level and on training year. RESULTS: CCCs from 211 pediatric fellowship programs provided data for 369 first, 336 second, and 331 third year fellows. Mean subcompetency milestone levels increased similarly among subspecialties for most EPAs compared with the reference, Adolescent Medicine. Mean subcompetency milestones mapped to each EPA and mean EPA LOS ratings generally increased by training year across all subspecialties. CONCLUSIONS: Subcompetency milestones levels mapped to each Common Subspecialty EPA and the EPA LOS ratings increase similarly across subspecialties and by training year, providing validity evidence for using EPA LOS to assess pediatric subspecialty trainee performance. This study supports the development of tools to facilitated the CCC evaluation process across all pediatric subspecialties.

Part 2: Components of a comprehensive diagnostic evaluation for autism spectrum disorders.

Part two in this series will discuss the American Academy of Pediatrics suggestions for the components of a comprehensive autism assessment. A better understanding about the critical components of a comprehensive assessment for ASD can help the PCP be an informed consumer of the information provided in report(s) from the referral source. This information also helps the PCP have a better understanding of what additional evaluations might be needed to provide a complete clinical picture of the child. This knowledge will guide the PCP in translating these evaluation components into a meaningful treatment plan for the child.

National Pediatric Experience With Virtual Interviews: Lessons Learned and Future Recommendations.

The coronavirus disease 2019 pandemic significantly impacted undergraduate and graduate medical education and created challenges that prevented a traditional approach to residency and fellowship recruitment and interviews. Early in the pandemic, the pediatric education community came together to support applicants and training programs and to foster an equitable recruitment process. We describe many of our community's innovations, including the use of virtual cafés to educate programs and highlight best practices for virtual recruitment and the use of regional webinars to highlight residency programs and provide information to applicants. Surveys of applicants and programs suggest that the virtual interview process worked well overall, with applicants and programs saving both time and money and programs maintaining a high rate of filling their positions. On the basis of this experience, we highlight the strengths and weaknesses of 3 potential models for future interview seasons. We close with a series of questions that need further investigation to create an effective and equitable recruitment process for the future.

Continued Supervision for the Common Pediatric Subspecialty Entrustable Professional Activities May Be Needed Following Fellowship Graduation.

PURPOSE: Entrustable professional activities (EPAs) are one approach to competency-based medical education (CBME), and 7 EPAs have been developed that address content relevant for all pediatric subspecialties. However, it is not known what level of supervision fellowship program directors (FPDs) deem necessary for graduation. The Subspecialty Pediatrics Investigator Network (SPIN) investigated FPD perceptions of the minimum level of supervision required for a trainee to successfully graduate. METHOD: In 2017, SPIN surveyed all FPDs of accredited fellowships for 14 subspecialties. For each EPA, the minimum supervision level for graduation (ranging from observation only to unsupervised practice) was set such that no more than 20% of FPDs would accept a lower level. RESULTS: The survey response rate was 82% (660/802). The minimum supervision level for graduation varied across the 7 EPAs from 2 (direct) to 4 (indirect for complex cases), with significant differences between EPAs. The percentage of FPDs desiring a lower minimum supervision level ranged from 3% to 17%. Compared with the 4 nonclinical EPAs (quality improvement, management, lead within the profession, scholarship), the 3 clinical EPAs (consultation, handover, lead a team) had higher minimum supervision graduation levels (P < .001), with less likelihood that an FPD would graduate a learner below their minimum level (P < .001). CONCLUSIONS: Consensus among FPDs across all pediatric subspecialties demonstrates the potential need for ongoing supervision for graduates in all 7 common pediatric subspecialty EPAs after fellowship. As CBME programs are implemented, processes and infrastructure to support new graduates are important considerations for leaders.

Achieving Entrustable Professional Activities During Fellowship.

BACKGROUND AND OBJECTIVES: Entrustable Professional Activities (EPAs) were developed to assess pediatric fellows. We previously showed that fellowship program directors (FPDs) may graduate fellows who still require supervision. How this compares with their expectations for entrustment of practicing subspecialists is unknown. METHODS: We surveyed US FPDs in 14 pediatric subspecialties through the Subspecialty Pediatrics Investigator Network between April and August 2017. For each of 7 common pediatric subspecialty EPAs, we compared the minimum level of supervision that FPDs required for graduation with the level they expected of subspecialists for safe and effective practice using the Friedman rank sum test and paired t test. We compared differences between subspecialties using linear regression. RESULTS: We collected data from 660 FPDs (response rate 82%). For all EPAs, FPDs did not require fellows to reach the level of entrustment for graduation that they expected of subspecialists to practice (P < .001). FPDs expected the least amount of supervision for the EPAs consultation and handovers. Mean differences between supervision levels for graduation and practice were smaller for clinical EPAs (consultation, handovers, lead a team) when compared with nonclinical EPAs (quality improvement, management, lead the profession and scholarship; P = .001) and were similar across nearly all subspecialties. CONCLUSIONS: Fellowship graduates may need continued development of clinical and nonclinical skills in their early practice period, underscoring a need for continued assessment and mentoring. Graduation readiness must be based on clear requirements, with alignment of FPD expectations and regulatory standards, to ensure quality care for patients.

Agreement of Program Directors With Clinical Competency Committees for Fellow Entrustment.

OBJECTIVES: Fellowship program directors (FPD) and Clinical Competency Committees (CCCs) both assess fellow performance. We examined the association of entrustment levels determined by the FPD with those of the CCC for 6 common pediatric subspecialty entrustable professional activities (EPAs), hypothesizing there would be strong correlation and minimal bias between these raters. METHODS: The FPDs and CCCs separately assigned a level of supervision to each of their fellows for 6 common pediatric subspecialty EPAs. For each EPA, we determined the correlation between FPD and CCC assessments and calculated bias as CCC minus FPD values for when the FPD was or was not a member of the CCC. In addition, we examined the effect of program size, FPD understanding of EPAs, and subspecialty on the correlations. Data were obtained in fall 2014 and spring 2015. RESULTS: A total of 1040 fellows were assessed in the fall and 1048 in the spring. In both periods and for each EPA, there was a strong correlation between FPD and CCC supervision levels (P < .001). The correlation was somewhat lower when the FPD was not a CCC member (P < .001). Overall bias in both periods was small. CONCLUSIONS: The correlation between FPD and CCC assignment of EPA supervision levels is strong. Although slightly weaker when the FPD is not a CCC member, bias is small, so this is likely unimportant in determining fellow entrustment level. The similar performance ratings of FPDs and CCCs support the validity argument for EPAs as competency-based assessment tools.

Triple risk: do difficult temperament and family conflict increase the likelihood of behavioral maladjustment in children born low birth weight and preterm?

OBJECTIVE: This study examined the impact of family conflict on internalizing and externalizing behavior at age 8 for children born low birth weight (LBW) and preterm (PT), with specific attention to the moderating role of early temperament. METHODS: The sample included 728 families enrolled in the longitudinal study of the Infant Health and Development Program. The study relied on maternal reports of child temperament at age 1, family conflict at age 6(1/2) years and 8 years, and child behavior at age 8 years. RESULTS: Children exposed to high levels of family conflict had more internalizing problems. Child temperament assessed in infancy moderated the impact of family conflict on externalizing but not internalizing problem behavior. DISCUSSION: LBW/PT children with a difficult temperament are more at risk for poor developmental outcomes, such as externalizing behavior problems, when exposed to family conflict than children with a less difficult temperament.

Incorporating the ACGME educational competencies into morbidity and mortality review conferences.

BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) mandates that residents be trained in six core educational competencies. Practice-based learning and improvement (PBLI), one of the six competencies, is defined as the investigation and evaluation of one's own patient care. Morbidity and Mortality Conference, a frequently used venue to review the clinical outcome of hospitalized patients, provides the opportunity to teach and assess PBLI. DESCRIPTION: We report an approach to Morbidity and Mortality Conference that includes a systematic analysis of the ACGME core competencies and their application to a clinical case, a regular review of the factors that defines high-quality patient care, and a focused discussion of the PBLI competency. EVALUATION: Preliminary data indicate that our residents preferred this revised method for conducting Morbidity and Mortality Conference. CONCLUSION: Our adaptation to Morbidity and Mortality Conference provides a systematic review of the core competencies and their relevance to clinical decision making, with the ultimate goal of improving patient care.

Parenting: linking impacts of interpartner conflict to preschool children's social behavior.

Family conflict is known to have detrimental impacts on the social development of young children. An important issue in counseling parents and the development of intervention for children is the extent to which other family environmental conditions are the path through which conflict impacts children's development. This study examined two maternal parenting behaviors (harsh discipline and warmth) that may alter the impact of interpartner conflict on child social development and behavior in a large (n = 440 girls, n = 451 boys) sample of ethnically diverse, low-income families of preschool children. Interpartner conflict was associated with poorer child social development and behavior problems. This study found that interpartner conflict increased harsh discipline, which resulted in poorer child social development. This study, however, found no evidence that interpartner conflict impacted child development through its impact on maternal warmth in that mothers experiencing conflict did not alter the level of warm parenting practices. These findings suggest that, when encountering families experiencing interpartner conflict, clinicians should not only direct families to interventions to lessen family conflict but also counsel them on the mechanism (harsh discipline) by which children are impacted by the conflict.