Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Level of Supervision for the Entrustable Professional Activities Common to General Pediatrics and the Subspecialties Decreases from Residency to Fellowship.

OBJECTIVES: To compare level of supervision (LOS) ratings of graduating pediatric residents with their assessments as fellows for the five Entrustable Professional Activities (EPAs) common to general pediatrics and the subspecialties and to determine if the difference between ratings from residency to fellowship is less for the QI and Practice Management EPAs, since the skills needed to perform these may be less context-dependent. METHODS: We compared ratings of graduating residents with their assessments as fellows using LOS data from two sequential EPA studies. RESULTS: There were 65 ratings from 41 residents at the first fellow assessment. At graduation, most residents needed little to no supervision for all EPAs with 94% (61/65) of ratings level four or five. In contrast, only 5/65 (8%) of the first fellow assessments were level four or five. The ratings difference for the QI and Practice Management EPAs was similar to the others. CONCLUSIONS: LOS ratings for the EPAs common to generalists and subspecialists reset as residents become fellows. There was no evidence that the QI and Practice Management EPAs are less context-dependent. This study provides additional validity evidence for using these LOS scales to assess trainees in pediatric residency and fellowship.

Child Health Needs and the Developmental-Behavioral Pediatrics Workforce Supply: 2020-2040.

Developmental-behavioral pediatrics (DBP) subspecialists care for children with complex neurodevelopmental and behavioral health conditions; additional roles include education and training, advocacy, and research. In 2023, there were 1.0 DBP subspecialists per 100 000 US children aged 0 to 17 years (range 0.0-3.8), with wide variability in DBP subspecialist distribution. Given the prevalence of DB conditions, the current workforce is markedly inadequate to meet the needs of patients and families. The American Board of Pediatrics Foundation led a modeling project to forecast the US pediatric subspecialty workforce from 2020 to 2040 using current trends in each subspecialty. The model predicts workforce supply at baseline and across alternative scenarios and reports results in headcount (HC) and HC adjusted for percent time spent in clinical care, termed "clinical workforce equivalent." For DBP, the baseline model predicts HC growth nationally (+45%, from 669 to 958), but these extremely low numbers translate to minimal patient care impact. Adjusting for population growth over time, projected HC increases from 0.8 to 1.0 and clinical workforce equivalent from 0.5 to 0.6 DBP subspecialists per 100 000 children aged 0 to 18 years by 2040. Even in the best-case scenario (+12.5% in fellows by 2030 and +7% in time in clinical care), the overall numbers would be minimally affected. These current and forecasted trends should be used to shape much-needed solutions in education, training, practice, policy, and workforce research to increase the DBP workforce and improve overall child health.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment.

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Clinician Diagnostic Certainty and the Role of the Autism Diagnostic Observation Schedule in Autism Spectrum Disorder Diagnosis in Young Children.

Importance: Autism spectrum disorder (ASD) affects 1 in 44 children. The Autism Diagnostic Observation Schedule (ADOS) is a semi-structured observation developed for use in research but is considered a component of gold standard clinical diagnosis. The ADOS adds time and cost to diagnostic assessments. Objective: To evaluate consistency between clinical diagnosis (index ASD diagnosis) and diagnosis incorporating the ADOS (reference standard ASD diagnosis) and to examine clinician and child factors that predict consistency between index diagnoses and reference standard diagnoses. Design, Setting, and Participants: This prospective diagnostic study was conducted between May 2019 and February 2020. Developmental-behavioral pediatricians (DBPs) made a diagnosis based on clinical assessment (index ASD diagnosis). The ADOS was then administered, after which the DBP made a second diagnosis (reference standard ASD diagnosis). DBPs self-reported diagnostic certainty at the time of the index diagnoses and reference standard diagnoses. The study took place at 8 sites (7 US and 1 European) that provided subspecialty assessments for children with concerns for ASD. Participants included children aged 18 months to 5 years, 11 months, without a prior ASD diagnosis, consecutively referred for possible ASD. Among 648 eligible children, 23 refused, 376 enrolled, and 349 completed the study. All 40 eligible DBPs participated. Exposures: ADOS administered to all child participants. Main Outcomes and Measures: Index diagnoses and reference standard diagnoses of ASD (yes/no). Results: Among the 349 children (279 [79.7%] male; mean [SD] age, 39.9 [13.4] months), index diagnoses and reference standard diagnoses were consistent for 314 (90%) (ASD = 250; not ASD = 64) and changed for 35. Clinician diagnostic certainty was the most sensitive and specific predictor of diagnostic consistency (area under curve = 0.860; P < .001). In a multilevel logistic regression, no child or clinician factors improved prediction of diagnostic consistency based solely on clinician diagnostic certainty at time of index diagnosis. Conclusions and Relevance: In this prospective diagnostic study, clinical diagnoses of ASD by DBPs with vs without the ADOS were consistent in 90.0% of cases. Clinician diagnostic certainty predicted consistency of index diagnoses and reference standard diagnoses. This study suggests that the ADOS is generally not required for diagnosis of ASD in young children by DBPs and that DBPs can identify children for whom the ADOS may be needed.

Benefits and Limitations of Virtual Recruitment: Perspectives From Subspeciality Directors.

BACKGROUND AND OBJECTIVES: Because of the coronavirus disease 2019 pandemic and recommendations from a range of leaders and organizations, the pediatrics subspecialty 2020 recruitment season was entirely virtual. Minimal data exist on the effect of this change to guide future strategies. The aim of this study was to understand the effects of virtual recruitment on pediatric subspecialty programs as perceived by program leaders. METHODS: This concurrent, triangulation, mixed-methods study used a survey that was developed through an iterative (3 cycles), consensus-building, modified Delphi process and sent to all pediatric subspecialty program directors (PSPDs) between April and May 2021. Descriptive statistics and thematic analysis were used, and a conceptual framework was developed. RESULTS: Forty-two percent (352 of 840) of PSPDs responded from 16 of the 17 pediatric (94%) subspecialties; 60% felt the virtual interview process was beneficial to their training program. A majority of respondents (72%) reported cost savings were a benefit; additional benefits included greater efficiency of time, more applicants per day, greater faculty involvement, and perceived less time away from residency for applicants. PSPDs reported a more diverse applicant pool. Without an in-person component, PSPDs worried about programs and applicants missing informative, in-person interactions and applicants missing hospital tours and visiting the city. A model based upon theory of change was developed to aid program considerations for future application cycles. CONCLUSIONS: PSPDs identified several benefits to virtual recruitment, including ease of accommodating increased applicants with a diverse applicant pool and enhanced faculty involvement. Identified limitations included reduced interaction between the applicant and the larger institution/city.

Pilot Study of Maternal Autoantibody-Related Autism.

OBJECTIVE: The objective of this study was to investigate the presence of maternal autoantibody-related autism spectrum disorder (MAR-ASD) in 2 geographically distinct DBPNet clinical sites (Pennsylvania and Arkansas). MAR-ASD is a biologically defined subtype of ASD that is defined by the presence of autoantibodies specific to proteins in the fetal brain and present in approximately 20% of a Northern California sample but has not been studied in other states. METHODS: Sixty-eight mothers of children with ASD were recruited from 2 DBPNet clinics and provided blood samples. Mothers also completed behavioral questionnaires about their children, and data from the child's clinical diagnostic assessment were abstracted. RESULTS: The mean age of mothers was 38.5 ± 6.1 years, and the mean age of children was 8.3 ± 2.7 years. MAR-ASD was present in 24% of the sample and similar across sites. Children of +MAR mothers had more severe autism symptoms as measured by Autism Diagnostic Observation Schedule comparison scores (W = 3604; p < 0.001) and the Social Communication Questionnaire (W = 4556; p < 0.001). There were no differences in IQ, adaptive function, or aberrant behavior. CONCLUSION: MAR-ASD is a subtype of autism that is present in similar frequencies across 3 states and related to autism severity.

Defining Developmental-Behavioral Pediatrics.

There is an insufficient number of specialty developmental-behavioral pediatrics (DBP) physicians, despite nearly 25% of children and adolescents having a developmental, learning, behavioral, or emotional problem. In the nearly 20 years since becoming a board-certified subspecialty, the definition of DBP clinical practice remains somewhat unclear. This lack of clarity likely contributes to recruitment challenges and workforce issues, and limited visibility of DBP among parents, other professionals, payors, and administrators. Defining DBP is therefore an important step in the survival and growth of the field. In this paper, we describe the methodology used to develop this definition along with the origins of DBP, the persistent challenges to defining its scope, what training in DBP involves, and what distinguishes DBP from other overlapping fields of medicine. We propose the following definition of DBP: developmental-behavioral pediatrics (DBP) is a board-certified, medical subspecialty that cares for children with complex and severe DBP problems by recognizing the multifaceted influences on the development and behavior of children and addressing them through systems-based practice and a neurodevelopmental, strength-based approach that optimizes functioning. Developmental behavioral pediatricians care for children from birth through young adulthood along a continuum including those suspected of, at risk for, or known to have developmental and behavioral disorders.

Fellow Entrustment for the Common Pediatric Subspecialty Entrustable Professional Activities Across Subspecialties.

OBJECTIVE: To determine the relationship between level of supervision (LOS) ratings for the Common Pediatric Subspecialty Entrustable Professional Activities (EPAs) with their associated subcompetency milestones across subspecialties and by fellowship training year. METHODS: Clinical Competency Committees (CCCs) in 14 pediatric subspecialties submitted LOS ratings for 6 Common Subspecialty EPAs and subcompetency milestone levels mapped to these EPAs. We examined associations between these subcompetency milestone levels and LOS ratings across subspecialty training year by fitting per-EPA linear mixed effects models, regressing LOS rating on milestone level and on training year. RESULTS: CCCs from 211 pediatric fellowship programs provided data for 369 first, 336 second, and 331 third year fellows. Mean subcompetency milestone levels increased similarly among subspecialties for most EPAs compared with the reference, Adolescent Medicine. Mean subcompetency milestones mapped to each EPA and mean EPA LOS ratings generally increased by training year across all subspecialties. CONCLUSIONS: Subcompetency milestones levels mapped to each Common Subspecialty EPA and the EPA LOS ratings increase similarly across subspecialties and by training year, providing validity evidence for using EPA LOS to assess pediatric subspecialty trainee performance. This study supports the development of tools to facilitated the CCC evaluation process across all pediatric subspecialties.

Achieving Entrustable Professional Activities During Fellowship.

BACKGROUND AND OBJECTIVES: Entrustable Professional Activities (EPAs) were developed to assess pediatric fellows. We previously showed that fellowship program directors (FPDs) may graduate fellows who still require supervision. How this compares with their expectations for entrustment of practicing subspecialists is unknown. METHODS: We surveyed US FPDs in 14 pediatric subspecialties through the Subspecialty Pediatrics Investigator Network between April and August 2017. For each of 7 common pediatric subspecialty EPAs, we compared the minimum level of supervision that FPDs required for graduation with the level they expected of subspecialists for safe and effective practice using the Friedman rank sum test and paired t test. We compared differences between subspecialties using linear regression. RESULTS: We collected data from 660 FPDs (response rate 82%). For all EPAs, FPDs did not require fellows to reach the level of entrustment for graduation that they expected of subspecialists to practice (P < .001). FPDs expected the least amount of supervision for the EPAs consultation and handovers. Mean differences between supervision levels for graduation and practice were smaller for clinical EPAs (consultation, handovers, lead a team) when compared with nonclinical EPAs (quality improvement, management, lead the profession and scholarship; P = .001) and were similar across nearly all subspecialties. CONCLUSIONS: Fellowship graduates may need continued development of clinical and nonclinical skills in their early practice period, underscoring a need for continued assessment and mentoring. Graduation readiness must be based on clear requirements, with alignment of FPD expectations and regulatory standards, to ensure quality care for patients.

National Pediatric Experience With Virtual Interviews: Lessons Learned and Future Recommendations.

The coronavirus disease 2019 pandemic significantly impacted undergraduate and graduate medical education and created challenges that prevented a traditional approach to residency and fellowship recruitment and interviews. Early in the pandemic, the pediatric education community came together to support applicants and training programs and to foster an equitable recruitment process. We describe many of our community's innovations, including the use of virtual cafés to educate programs and highlight best practices for virtual recruitment and the use of regional webinars to highlight residency programs and provide information to applicants. Surveys of applicants and programs suggest that the virtual interview process worked well overall, with applicants and programs saving both time and money and programs maintaining a high rate of filling their positions. On the basis of this experience, we highlight the strengths and weaknesses of 3 potential models for future interview seasons. We close with a series of questions that need further investigation to create an effective and equitable recruitment process for the future.

Continued Supervision for the Common Pediatric Subspecialty Entrustable Professional Activities May Be Needed Following Fellowship Graduation.

PURPOSE: Entrustable professional activities (EPAs) are one approach to competency-based medical education (CBME), and 7 EPAs have been developed that address content relevant for all pediatric subspecialties. However, it is not known what level of supervision fellowship program directors (FPDs) deem necessary for graduation. The Subspecialty Pediatrics Investigator Network (SPIN) investigated FPD perceptions of the minimum level of supervision required for a trainee to successfully graduate. METHOD: In 2017, SPIN surveyed all FPDs of accredited fellowships for 14 subspecialties. For each EPA, the minimum supervision level for graduation (ranging from observation only to unsupervised practice) was set such that no more than 20% of FPDs would accept a lower level. RESULTS: The survey response rate was 82% (660/802). The minimum supervision level for graduation varied across the 7 EPAs from 2 (direct) to 4 (indirect for complex cases), with significant differences between EPAs. The percentage of FPDs desiring a lower minimum supervision level ranged from 3% to 17%. Compared with the 4 nonclinical EPAs (quality improvement, management, lead within the profession, scholarship), the 3 clinical EPAs (consultation, handover, lead a team) had higher minimum supervision graduation levels (P < .001), with less likelihood that an FPD would graduate a learner below their minimum level (P < .001). CONCLUSIONS: Consensus among FPDs across all pediatric subspecialties demonstrates the potential need for ongoing supervision for graduates in all 7 common pediatric subspecialty EPAs after fellowship. As CBME programs are implemented, processes and infrastructure to support new graduates are important considerations for leaders.

Agreement of Program Directors With Clinical Competency Committees for Fellow Entrustment.

OBJECTIVES: Fellowship program directors (FPD) and Clinical Competency Committees (CCCs) both assess fellow performance. We examined the association of entrustment levels determined by the FPD with those of the CCC for 6 common pediatric subspecialty entrustable professional activities (EPAs), hypothesizing there would be strong correlation and minimal bias between these raters. METHODS: The FPDs and CCCs separately assigned a level of supervision to each of their fellows for 6 common pediatric subspecialty EPAs. For each EPA, we determined the correlation between FPD and CCC assessments and calculated bias as CCC minus FPD values for when the FPD was or was not a member of the CCC. In addition, we examined the effect of program size, FPD understanding of EPAs, and subspecialty on the correlations. Data were obtained in fall 2014 and spring 2015. RESULTS: A total of 1040 fellows were assessed in the fall and 1048 in the spring. In both periods and for each EPA, there was a strong correlation between FPD and CCC supervision levels (P < .001). The correlation was somewhat lower when the FPD was not a CCC member (P < .001). Overall bias in both periods was small. CONCLUSIONS: The correlation between FPD and CCC assignment of EPA supervision levels is strong. Although slightly weaker when the FPD is not a CCC member, bias is small, so this is likely unimportant in determining fellow entrustment level. The similar performance ratings of FPDs and CCCs support the validity argument for EPAs as competency-based assessment tools.

Autism Speaks Toolkits: Resources for Busy Physicians.

Given the increased prevalence of autism spectrum disorders (ASD), it is likely that busy primary care providers (PCP) are providing care to individuals with ASD in their practice. Autism Speaks provides a wealth of educational, medical, and treatment/intervention information resources for PCPs and families, including at least 32 toolkits. This article serves to familiarize PCPs and families on the different toolkits that are available on the Autism Speaks website. This article is intended to increase physicians' knowledge on the issues that families with children with ASD frequently encounter, to increase their ability to share evidence-based information to guide treatment and care for affected families in their practice.

Financial and employment impact of intellectual disability on families of children with autism.

Families of children with autism spectrum disorder (ASD) endure significant financial and employment burden because of their children's numerous needed services. The study objective is to describe additional impact on families of children with intellectual disability (ID) in addition to ASD. The study is a secondary data analysis of the 2009-2010 National Survey of Children with Special Health Care Needs. Children whose parents answered "yes" when asked whether their child had ASD or ID were classified as having ASD alone (unweighted n = 2,406), ID alone (unweighted n = 1,363), or both ASD/ID (unweighted n = 620). Bivariate and multivariate analyses compared study outcomes of family financial and caregiver burden using ASD as the reference group. All analyses were weighted using person-level estimates. Of children with ASD, 24% also had ID. More than half of caregivers of children with ASD/ID reported financial difficulty (52%) and having to stop work to care for their child (51%). Compared with ASD alone, caregivers of children with both ASD/ID were more likely to report financial difficulty (aOR 1.65, 95% CI 1.10-2.46), cutting work hours (aOR 1.43, 95% CI .98-2.08), and stop working (aOR 2.32, 95% CI 1.57-3.43). No differences were found between caregivers of children with ASD only and ID only. We conclude that having ID in addition to ASD may be associated with greater negative impact on family financial and employment burden. Recognition of ID in addition to ASD is important to tailor the clinical approach and sufficiently support families.

Maternal low- and high-depressive symptoms and safety concerns for low-income preschool children.

OBJECTIVES: Our objective was to examine the relationship between low- and high-level depressive symptoms in mothers' and children's risks in the areas of home and car safety, monitoring, and exposure to violence. METHODS: Participants included 978 mothers of preschool-age children who were interviewed about their home environment and screened for maternal depression. RESULTS: Whereas only 5.7% scored at high depressive levels on the screen, another 21.3% scored at low depressive levels. Logistic regression analyses controlling for demographics revealed that children were significantly more likely to experience home safety risks in 6 of 7 areas when mothers reported either low or high levels of depressive symptoms. CONCLUSIONS: Results suggest that children whose mothers experience even low-level depressive symptoms are at increased risk for safety problems in the home environment, pointing to the need for screening and interventions to reduce the risk of injury.