Thresholds and interpretations: How clinical competency committees identify pediatric residents with performance concerns.

BACKGROUND: Clinical competency committee (CCC) identification of residents with performance concerns is critical for early intervention. METHODS: Program directors and 94 CCC members at 14 pediatric residency programs responded to a written survey prompt asking them to describe how they identify residents with performance concerns. Data was analyzed using thematic analysis. RESULTS: Six themes emerged from analysis and were grouped into two domains. The first domain included four themes, each describing a path through which residents could meet or exceed a concern threshold:1) written comments from rotation assessments are foundational in identifying residents with performance concerns, 2) concerning performance extremes stand out, 3) isolated data points may accumulate to raise concern, and 4) developmental trajectory matters. The second domain focused on how CCC members and program directors interpret data to make decisions about residents with concerns and contained 2 themes: 1) using norm- and/or criterion-referenced interpretation, and 2) assessing the quality of the data that is reviewed. CONCLUSIONS: Identifying residents with performance concerns is important for their education and the care they provide. This study delineates strategies used by CCC members across several programs for identifying these residents, which may be helpful for other CCCs to consider in their efforts.

Design and Evaluation of an Interprofessional Preceptor Development Mini-Fellowship Program.

INTRODUCTION: Health professions preceptors require skills and knowledge to effectively meet the educational needs of interprofessional students in clinical environments. We implemented a mini-fellowship program to enhance the knowledge, skills, and self-efficacy of preceptors teaching students and applying quality improvement (QI) methods across disciplines and patient care settings. METHOD: The design, implementation, and evaluation of the program were informed by the faculty development literature, principles of adult learning, and preceptor needs. The 3-day program included workshops on curriculum design, clinical teaching methods, QI, social determinants of health, cultural humility, and interprofessional teamwork. Quantitative and qualitative evaluation methods were used including preprogram and postprogram knowledge and self-efficacy surveys, along with end-of-session and program evaluations. RESULTS: Five annual cohorts involving 41 preceptors with varied demographics, professions, and clinical practices completed the mini-fellowship program. Participants' percentage of items answered correctly on a QI knowledge test increased from 79.2% (pretest) to 85.5% (post-test), a gain of 6.3% (90% CI: 2.9-9.7%; P < .003). The average QI self-efficacy scores improved from 2.64 to 3.82, a gain of 1.18 points on a five-point scale (P < .001). The average education/teaching self-efficacy increased from 2.79 to 3.80 on a five-point scale (P < .001). Ultimately, 94% would recommend the program to other preceptors. DISCUSSION: An interprofessional preceptor development program designed to train clinicians to effectively teach in the clinical setting and to conduct QI projects with students was achievable and effective. This program can serve as a model for academic centers charged with training future health care workers and supporting their community-based preceptors' training needs.

Caregiver Inclusivity and Empowerment During Family-Centered Rounds.

OBJECTIVE: Despite widespread adoption of family-centered rounds, few have investigated differences in the experience of family-centered rounds by family race and ethnicity. The purpose of this study was to explore racial and ethnic differences in caregiver perception of inclusion and empowerment during family-centered rounds. METHODS: We identified eligible caregivers of children admitted to the general pediatrics team through the electronic health record. Surveys were completed by 99 caregivers (47 non-Latinx White and 52 Black, Latinx, or other caregivers of color). To compare agreement with statements of inclusivity and empowerment, we used the Wilcoxon rank sum test in unadjusted analyses and linear regression for the adjusted analyses. RESULTS: Most (91%) caregivers were satisfied or extremely satisfied with family-centered rounds. We found no differences by race or ethnicity in statements of satisfaction or understanding family-centered rounds content. However, in both unadjusted and adjusted analyses, we found that White caregivers more strongly agreed with the statements "I felt comfortable participating in rounds," "I had adequate time to ask questions during rounds," and "I felt a valued member of the team during rounds" compared with Black, Latinx, and other caregivers of color. CONCLUSIONS: Congruent with studies of communication in other settings, caregivers of color may experience barriers to inclusion in family-centered rounds, such as medical team bias, less empathic communication, and shorter encounters. Future studies are needed to better understand family-centered rounds disparities and develop interventions that promote inclusive rounds.

Initial Modified Pulmonary Index Score Predicts Hospital Length of Stay for Asthma Subjects Admitted to the Pediatric Intensive Care Unit.

BACKGROUND: Scoring systems are frequently used to assess the severity of pediatric asthma exacerbations. The modified pulmonary index score (MPIS) has been found to be highly correlated with length of stay (LOS) in the pediatric intensive care unit (PICU). We sought to evaluate the use of the MPIS to predict hospital LOS for patients admitted to our PICU. METHODS: We retrospectively reviewed the medical records of pediatric asthma subjects aged 2-17 y admitted to our PICU between June 2014 and November 2017. We divided subjects a priori into 3 groups (low: MPIS 0-5; medium: MPIS 6-9; high: MPIS ≥ 10) based upon each subject's first MPIS documented in the PICU. Hospital LOS, PICU LOS, time on continuous albuterol, and increased respiratory support were compared between groups. RESULTS: 143 subjects were included. There were no differences for demographics, medical history, cause of exacerbations, or mean heart rate between groups. There were significant differences between groups for mean breathing frequency (P < .001), [Formula: see text] (P = .01), and [Formula: see text] (P < .004). There were significant differences between groups for route of admission (P = .02), high-flow nasal cannula use (P < .001), and use of a helium-oxygen mixture (P < .001). There were significant differences between groups for median hospital LOS (1.2 vs 2.3 vs 3.4 d, P < .001), PICU LOS (0.39 vs 1.3 vs 2 d, P < .001), and time on continuous albuterol (7.4 vs 20.6 vs 34.7 h, P < .001). After adjusting for demographics and medical history, the incidence risk ratio for hospital LOS was 2.09 for PICU admission for an MPIS of 6-9 and 2.68 for an MPIS ≥ 10 when compared to an MPIS < 6. CONCLUSIONS: The MPIS thresholds used in our pathway appropriately predicted LOS in our cohort of subjects with asthma admitted to the PICU. Higher MPIS was associated with increased hospital LOS, PICU LOS, and time on continuous albuterol.

Faculty and Resident Perspectives on Teaching Pediatric Evidence-Based Medicine in the Clinical Setting: A Qualitative Study.

PURPOSE: To describe how pediatric educators effectively teach evidence-based medicine (EBM) in the clinical setting. Secondarily, to identify barriers hindering effective practice and teaching of EBM and strategies to overcome these barriers. METHODS: The authors conducted a cross-sectional multi-institutional qualitative study from July 2016 to December 2017 in which they interviewed pediatric educators across many subspecialties who were identified as exemplary teachers of EBM at 3 academic pediatric residency programs. Pediatric residents who had recently worked with these faculty members were also interviewed to allow triangulation between participants. Qualitative analysis was complete once saturation was achieved. RESULTS: Twenty-six pediatric educators identified as exemplary teachers of EBM and 10 residents who worked with those educators participated in the study. Thirteen explicit teaching strategies and 2 implicit teaching strategies, namely disclosure of uncertainty and role modeling, were identified. Barriers to practicing clinical EBM included balancing patient responsibilities, inadequate time, and personal knowledge. Barriers to teaching clinical EBM were inadequate time and learner engagement. To overcome these barriers, faculty limit and focus teaching points, attempt to make EBM relevant to patient care, and incorporate follow-up strategies. CONCLUSIONS: Numerous teaching strategies are available to faculty to improve the clinical teaching of EBM and to overcome commonly encountered clinical EBM barriers. Familiarity with these clinical EBM teaching strategies can be used for faculty development and to enhance the teaching of EBM to learners.

Milestone Ratings and Supervisory Role Categorizations Swim Together, but Is the Water Muddy?

OBJECTIVE: This single-specialty, multi-institutional study aimed to determine 1) the association between milestone ratings for individual competencies and average milestone ratings (AMRs) and 2) the association between AMRs and recommended supervisory role categorizations made by individual clinical competency committee (CCC) members. METHODS: During the 2015-16 academic year, CCC members at 14 pediatric residencies reported milestone ratings for 21 competencies and recommended supervisory role categories (may not supervise, may supervise in some settings, may supervise in all settings) for residents they reviewed. An exploratory factor analysis of competencies was conducted. The associations among individual competencies, the AMR, and supervisory role categorizations were determined by computing bivariate correlations. The relationship between AMRs and recommended supervisory role categorizations was examined using an ordinal mixed logistic regression model. RESULTS: Of the 155 CCC members, 68 completed both milestone assignments and supervision categorizations for 451 residents. Factor analysis of individual competencies controlling for clustering of residents in raters and sites resulted in a single-factor solution (cumulative variance: 0.75). All individual competencies had large positive correlations with the AMR (correlation coefficient: 0.84-0.93), except for two professionalism competencies (Prof1: 0.63 and Prof4: 0.65). When combined across training year and time points, the AMR and supervisory role categorization had a moderately positive correlation (0.56). CONCLUSIONS: This exploratory study identified a modest correlation between average milestone ratings and supervisory role categorization. Convergence of competencies on a single factor deserves further exploration, with possible rater effects warranting attention.

A Respiratory Therapist-Driven Asthma Pathway Reduced Hospital Length of Stay in the Pediatric Intensive Care Unit.

BACKGROUND: Asthma is a common reason for admissions to the pediatric intensive care unit (PICU). Since June 2014, our institution has used a pediatric asthma clinical pathway for all patients, including those in PICU. The pathway promotes respiratory therapist-driven bronchodilator weaning based on the Modified Pulmonary Index Score (MPIS). This pathway was associated with decreased hospital length of stay (LOS) for all pediatric asthma patients; however, the effect on PICU patients was unclear. We hypothesized that the implementation of a pediatric asthma pathway would reduce hospital LOS for asthmatic patients admitted to the PICU. METHODS: We retrospectively reviewed the medical records of all pediatric asthma subjects 2-17 y old admitted to our PICU before and after pathway initiation. Primary outcome was hospital LOS. Secondary outcomes were PICU LOS and time on continuous albuterol. Data were analyzed using the chi-square test for categorical data, the t test for normally distributed data, and the Mann-Whitney test for nonparametric data. RESULTS: A total of 203 eligible subjects (49 in the pre-pathway group, 154 in the post group) were enrolled. There were no differences between groups for age, weight, gender, home medications, cause of exacerbation, medical history, or route of admission. There were significant decreases in median (interquartile range) hospital LOS (4.4 [2.9-6.6] d vs 2.7 [1.6-4.0] d, P < .001), median PICU LOS (2.1 [1.3-4.0] d vs 1.6 [0.8-2.4] d, P = .003), and median time on continuous albuterol (39 [25-85] h vs 27 [13-42] h, P = .001). Significantly more subjects in the post-pathway group were placed on high-flow nasal cannula (32% vs 6%, P = .001) or noninvasive ventilation (10% vs 4%, P = .02). CONCLUSION: The implementation of an asthma pathway was associated with decreased hospital LOS, PICU LOS, and time on continuous albuterol. There was also an increase in the use of high-flow nasal cannula and noninvasive ventilation after the implementation of this clinical pathway.

Key Factors in Clinical Competency Committee Members' Decisions Regarding Residents' Readiness to Serve as Supervisors: A National Study.

PURPOSE: Entrustment has become a popular assessment framework in recent years. Most research in this area has focused on how frontline assessors determine when a learner can be entrusted. However, less work has focused on how these entrustment decisions are made. The authors sought to understand the key factors that pediatric residency program clinical competency committee (CCC) members consider when recommending residents to a supervisory role. METHOD: CCC members at 14 pediatric residency programs recommended residents to one of five progressive supervisory roles (from not serving as a supervisory resident to serving as a supervisory resident in all settings). They then responded to a free-text prompt, describing the key factors that led them to that decision. The authors analyzed these responses, by role recommendation, using a thematic analysis. RESULTS: Of the 155 CCC members at the participating programs, 84 completed 769 supervisory role recommendations during the 2015-2016 academic year. Four themes emerged from the thematic analysis: (1) Determining supervisory ability follows from demonstrated trustworthiness; (2) demonstrated performance matters, but so does experience; (3) ability to lead a team is considered; and (4) contextual considerations external to the resident are at play. CONCLUSIONS: CCC members considered resident and environmental factors in their summative entrustment decision making. The interplay between these factors should be considered as CCC processes are optimized and studied further.

Improving the Efficiency of Care for Pediatric Patients Hospitalized With Asthma.

BACKGROUND: Asthma exacerbations are a leading cause of hospitalization among children. Despite the existence of national pediatric asthma guidelines, significant variation in care persists. At Duke Children's Hospital, we determined that our average length of stay (ALOS) and cost for pediatric asthma admissions exceeded that of our peers. Our aim was to reduce the ALOS of pediatric patients hospitalized with asthma from 2.9 days to 2.6 days within 12 months by implementing an asthma pathway within our new electronic health record. METHODS: We convened a multidisciplinary committee charged with reducing variability in practice, ALOS, and cost of inpatient pediatric asthma care, while adhering to evidence-based guidelines. Interventions were tested through multiple "plan-do-study-act" cycles. Control charts of the ALOS were constructed and annotated with interventions, including testing of an asthma score, implementation of order sets, use of a respiratory therapy-driven albuterol treatment protocol, and provision of targeted education. Order set usage was audited as a process measure. Readmission rates were monitored as a balancing measure. RESULTS: The ALOS of pediatric patients hospitalized with asthma decreased significantly from 2.9 days to 2.3 days. Comparing baseline with intervention variable direct cost data revealed a savings of $1543 per case. Improvements occurred in the context of high compliance with the asthma pathway order sets. Readmission rates remained stable throughout the study period. CONCLUSIONS: Implementation of an asthma care pathway based on the electronic health record improved the efficiency and variable direct costs of hospital care, reduced variability in practice, and ensured adherence to high-quality national guidelines.

Milestone-Based Assessments Are Superior to Likert-Type Assessments in Illustrating Trainee Progression.

BACKGROUND: The Pediatrics Milestone Project uses behavioral anchors, narrative descriptions of observable behaviors, to describe learner progression through the Accreditation Council for Graduate Medical Education competencies. Starting June 2014, pediatrics programs were required to submit milestone reports for their trainees semiannually. Likert-type scale assessment tools were not designed to inform milestone reporting, creating a challenge for Clinical Competency Committees. OBJECTIVE: To determine if milestone-based assessments better stratify trainees by training level compared to Likert-type assessments. METHODS: We compared assessment results for 3 subcompetencies after changing from a 5-point Likert scale to milestone-based behavioral anchors in July 2013. Program leadership evaluated the new system by (1) comparing PGY-1 mean scores on Likert-type versus milestone-based assessments; and (2) comparing mean scores on the Likert-type versus milestone-based assessments across PGY levels. RESULTS: Mean scores for PGY-1 residents were significantly higher on the prior year's Likert-type assessments than milestone-based assessments for all 3 subcompetencies (P < .01). Stratification by PGY level was not observed with Likert-type assessments (eg, interpersonal and communication skills 1 [ICS1] mean score for PGY-1, 3.99 versus PGY-3, 3.98; P  =  .98). In contrast, milestone-based assessments demonstrated stratification by PGY level (eg, the ICS1 mean score was 3.06 for PGY-1, 3.83 for PGY-2, and 3.99 for PGY-3; P < .01 for PGY-1 versus PGY-3). Significantly different means by trainee level were noted across 21 subcompetencies on milestone-based assessments (P < .01 for PGY-1 versus PGY-3). CONCLUSIONS: Initial results indicate milestone-based assessments stratify trainee performance by level better than Likert-type assessments. Average PGY-level scores from milestone-based assessments may ultimately provide guidance for determining whether trainees are progressing at the expected pace.

Using an innovative curriculum evaluation tool to inform program improvement: the clinical skills fair.

BACKGROUND: Program evaluation is important for assessing the effectiveness of the residency curriculum. Limited resources are available, however, and curriculum evaluation processes must be sustainable and well integrated into program improvement efforts. INTERVENTION: We describe the pediatric Clinical Skills Fair, an innovative method for evaluating the effectiveness of residency curriculum through assessment of trainees in 2 domains: medical knowledge/patient care and procedure. Each year from 2008 to 2011, interns completed the Clinical Skills Fair as rising interns in postgraduate year (PGY)-1 (R1s) and again at the end of the year, as rising residents in PGY-2 (R2s). Trainees completed the Clinical Skills Fair at the beginning and end of the intern year for each cohort to assess how well the curriculum prepared them to meet the intern goals and objectives. RESULTS: Participants were 48 R1s and 47 R2s. In the medical knowledge/patient care domain, intern scores improved from 48% to 65% correct (P < .001). Significant improvement was demonstrated in the following subdomains: jaundice (41% to 65% correct; P < .001), fever (67% to 94% correct; P < .001), and asthma (43% to 62% correct; P  =  .002). No significant change was noted within the arrhythmia subdomain. There was significant improvement in the procedure domain for all interns (χ(2)  =  32.82, P < .001). CONCLUSIONS: The Clinical Skills Fair is a readily implemented and sustainable method for our residency program curriculum assessment. Its feasibility may allow other programs to assess their curriculum and track the impact of programmatic changes; it may be particularly useful for program evaluation committees.