A Qualitative Analysis of Assessment and Evaluation Practices Within the Individualized Curriculum.

OBJECTIVE: The Accreditation Council of Graduate Medical Education requires an "individualized curriculum" (IC) in pediatric residency. A shared understanding across programs of methods to evaluate the IC is lacking. We explored pediatric program leaders' perceptions of assessment and evaluation within the IC to further understand and inform most useful practices. METHODS: We conducted a phenomenology study using semi-structured interviews to 1) determine what IC assessment and evaluation strategies are utilized in pediatric residency programs, and 2) explore program leaders' perceptions of the feasibility and value of assessment and evaluation in the IC. We recruited a purposive sample of leaders from 15 pediatric residency programs of various sizes and regions. Interviews were recorded and transcribed. Data were analyzed to produce themes. RESULTS: Three themes arose from our analysis: 1) Systematic assessment of the IC, though desired, is not robust in pediatric residency training; 2) Program differences present unique barriers and facilitators that impact the feasibility of assessment within the IC; and 3) Meaningfulness of assessment in the IC varies by stakeholder. Themes reflected perspectives on individual learner assessment and program evaluation. CONCLUSIONS: While systematic assessment and evaluation of the IC in pediatric residency training is lacking, program leaders desire feasible strategies that are meaningful to the resident, the program, and its leadership team. Leveraging current assessment and evaluation tools and aligning new assessment strategies could promote the integration of IC assessment with other assessments, minimizing burden. More structured IC assessment and evaluation could inform how to best achieve curricular goals of the IC.

Effect of implementation of a colorimetric quantitative blood loss system for postpartum hemorrhage.

OBJECTIVE: To evaluate if the implementation of a colorimetric quantitative blood loss (QBL) system during cesarean delivery improves clinical outcomes. METHODS: We conducted a retrospective cohort analysis after cesarean section before and after implementation of the Triton based colorimetric QBL system. Prevalence of postpartum hemorrhage, amount of blood products transfused, length of hospitalization, and rates of intensive care unit (ICU) admission were compared. RESULTS: A total of 2221 patients were included. There were 1192 patients in the pre-intervention group and 1029 patients in the post-intervention group. There was no significant difference between groups in the prevalence of postpartum hemorrhage (8.6% vs 9.3%, P = 0.57), amount of packed red blood cells (pRBCs) transfused (45 vs 30, P = 0.41) or average length of hospital stay in days (3.0 vs 3.0, P = 0.37). There was a statistically significant decrease in ICU admissions between the pre- and post-intervention groups (2.2% vs 1.0%, P = 0.02). CONCLUSION: There was no effect of implementation of the colorimetric QBL application system on diagnosis of postpartum hemorrhage, amount of blood products transfused, or length of hospital stay. Although a significant decrease in ICU admissions was observed, we could not determine if these transfers were hemorrhage related.

Reduction in Cesarean Delivery Rates Associated With a State Quality Collaborative in Maryland.

OBJECTIVE: To assess the extent to which hospitals participating in the MDPQC (Maryland Perinatal-Neonatal Quality Care Collaborative) to reduce primary cesarean deliveries adopted policy and practice changes and the association of this adoption with state-level cesarean delivery rates. METHODS: This prospective evaluation of the MDPQC includes 31 (97%) of the birthing hospitals in the state, which all voluntarily participated in the 30-month collaborative from June 2016 to December 2018. Hospital teams agreed to implement practices from the "Safe Reduction of Primary Cesarean Births" patient safety bundle, developed by the Council on Patient Safety in Women's Health Care. Each hospital's implementation of practices in the bundle was measured through surveys of team leaders at 12 months and 30 months. Half-yearly cesarean delivery rates were calculated from aggregate birth certificate data for each hospital, and differences in rates between the 6 months before the collaborative (baseline) and the 6 months afterward (endline) were tested for statistical significance. RESULTS: Among the 26 bundle practices that were assessed, participating hospitals reported having a median of seven practices (range 0-23) already in place before the collaborative and implementing a median of four (range 0-17) new practices during the collaborative. Across the collaborative, the cesarean delivery rates decreased from 28.5% to 26.9% (P=.011) for all nulliparous term singleton vertex births and from 36.1% to 31.3% (P<.001) for nulliparous, term, singleton, vertex inductions. Five hospitals had a statistically significant decrease in nulliparous, term, singleton, vertex cesarean delivery rates and four had a significant increase. Nulliparous, term, singleton, vertex cesarean delivery rates were significantly lower across hospitals that implemented more practices in the "Response" domain of the bundle. CONCLUSION: The MDPQC was associated with a statewide reduction in cesarean delivery rates for nulliparous, term, singleton, vertex births.

Burnout and Association With Resident Performance as Assessed by Pediatric Milestones: An Exploratory Study.

INTRODUCTION: Prior work demonstrating that burnout is associated with decreased performance in medical trainees has relied on self-report and/or single-site studies. We explored the relationship between burnout status and Milestones-based scores in pediatric residents nationally. METHODS: In April to June 2016, we confidentially surveyed residents using the Maslach Burnout Inventory. Separately, programs submitted resident Milestones scores in June 2016. We examined the relationship between burnout and performance as assessed by Milestones scores for each domain of competence. We performed multivariate analysis to determine which components of burnout (depersonalization [DP], emotional exhaustion, and lack of personal accomplishment [PA]) were most impactful. RESULTS: About 1494 of 2368 (63%) residents at 32 programs completed the Maslach Burnout Inventory and had Milestones scores submitted. Residents who scored positive for burnout scored lower in all Milestones domains. Subgroup analysis demonstrated that this association was only significant (P < .05) in the post-graduate year 1 (PGY1) categorical pediatric cohort. In the PGY1 residents (n = 442), those positive for burnout had lower Milestones scores in patient care (PC) (2.78 vs 2.98), systems-based practice (2.69 vs 2.87), practice-based learning and improvement (2.77 vs 2.93), professionalism (3.09 vs 3.24), and interpersonal and communication skills (2.95 vs 3.12), but not medical knowledge. Multivariate analysis demonstrated that, in PGY1 residents, lower PC score was associated with lower PA and higher DP. CONCLUSIONS: Burnout is associated with decreased Milestones performance for pediatric PGY1 residents. DP and low PA were associated with lower PC scores in PGY1 residents. Future research should address whether strategies to mitigate burnout improve PGY1 performance.

Describing Overarching Curricular Goals for Individualized Education.

Phenomenon: The phenomenon of individualized education, an essential component of competency-based medical education, addresses individual learner needs while working toward standardized learning outcomes. One challenge with broadly implementing individualized education is the lack of a pragmatic operational definition. To formalize expectations for individualized education, the Accreditation Council of Graduate Medical Education in 2013 began requiring six months of individualized curriculum (IC) during pediatric residency; however, there is not a national standard of formal curricular goals for the IC as an educational entity. Examining and describing the overarching curricular goals of IC could provide a framework for discourse about and further study of individualized education in medicine across disciplines and the continuum of medical education. Thus, we aimed to describe the phenomenon of individualized education through the lens of the goals of the IC in pediatric residency in the United States. Approach: In 2017, a purposeful sample of Pediatric Residency leaders were recruited to represent a diverse sample of program sizes, regions of the country, and importance of the IC to the program leadership. They completed an online survey with open-ended questions describing formal and implicit goals of their program's IC. The authors analyzed responses initially using conventional content analysis, then investigated whether the themes for program goals aligned with any existing educational theory. The concepts and language aligned with the principles of self-determination theory (SDT); therefore, the IC goals were subsequently grouped using the SDT domains of relatedness, autonomy, and competence. A focus group with a subset of survey respondents was conducted for member checking and elaboration of concepts. Findings: Program leaders from a diverse sample of 36 programs participated in the survey and a subset of 11 programs participated in the focus group. The common goals across all programs are listed in parentheses and organized by domains of SDT: 1)Relatedness goals (engage in mentorship, select a career) cultivate resident's professional identity based on their desired future career path; 2)autonomy goals (create a learning plan, practice accountability) help residents plan their path; and fulfillment of their plans lead to 3)competence goals (develop targeted clinical skills and knowledge, ensure comprehensive exposure, address learning gaps), ensuring they have a comprehensive skill set for their chosen identity. Insights: This study provides a framework to describe the phenomenon of individualized education through the lens of IC curricular goals in pediatric residency. The goals for IC that emerged from this study serve as a pragmatic framework for implementation of individualized education. They provide a common language and structure to promote more rigorous and collaborative study of individualized education across programs, disciplines, and settings in medicine. They may function as a roadmap for learners to navigate educational activities and for programs to help shape the experiences of their learners and examine outcomes of individualized education in their programs. The framework can also help individual pediatric residency programs structure improvements to their IC. Disciplines beyond pediatrics may also use this framework to better structure elective experiences to capitalize on the benefits of individual education.

Use of Individualized Learning Plans to Facilitate Feedback Among Medical Students.

THEORY: Self-regulated learning theory suggests that individualized learning plans can benefit medical trainees by providing a structured means of goal setting, self-monitoring, and self-evaluation. External feedback also plays an important role in affecting learner motivations, perceptions, and self-evaluations. Accordingly, having learners share individualized learning plans with preceptors might promote self-regulated learning by helping align the feedback they receive with their learning goals. Hypothesis: We hypothesized having medical students share individualized learning plans with attendings and residents would improve the quality of the feedback they received, increase the likelihood that feedback correlated to their learning goals, and improve their perceptions of feedback received. Method: In this multisite study, third-year medical students on their pediatric clerkship created individualized learning plans and shared them with residents and attendings by writing a learning goal on at least one of their required faculty feedback forms. The quality of feedback on forms with versus without a learning goal written on top was scored using a validated scoring tool and compared using a Wilcoxon signed-ranks test, and the frequency with which feedback directly correlated to a student learning goal on forms with versus without a learning goal written on top was compared using a chi-square test. Students completed a post-clerkship survey rating the quality of feedback and teaching they received, perceptions of the individualized learning plans, progress toward achieving learning goals, and whether or not they received teaching and/or feedback related to learning goals. Results: Thirty-six students completed a total of 108 learning goals and 181 feedback forms, of which 42 forms (23.2%) had a learning goal written on top. The mean (SD) feedback score between forms with [3.9 (0.9)] versus without [3.6 (0.6)] a learning goal written on top was not different (p = .113). Feedback on forms with a learning goal written on top was more likely to correlate to a student learning goal than feedback on forms without a learning goal (92.9% vs 23.0% respectively, p < .001). Student perceptions of the usefulness of learning goals did not differ between students who reported receiving teaching or feedback related to a learning goal and those who did not. Conclusions: Sharing individualized learning plans with preceptors helped align feedback with learning goals but did not affect the quality of feedback. Further research should examine the bidirectional relationship between individualized learning plans and feedback in light of other contextual and interpersonal factors.

Anterior trunk muscle shows mix of axial and appendicular developmental patterns.

BACKGROUND: Skeletal muscle in the trunk derives from the somites, paired segments of paraxial mesoderm. Whereas axial musculature develops within the somite, appendicular muscle develops following migration of muscle precursors into lateral plate mesoderm. The development of muscles bridging axial and appendicular systems appears mixed. RESULTS: We examine development of three migratory muscle precursor-derived muscles in zebrafish: the sternohyoideus (SH), pectoral fin (PF), and posterior hypaxial (PHM) muscles. We show there is an anterior to posterior gradient to the developmental gene expression and maturation of these three muscles. SH muscle precursors exhibit a long delay between migration and differentiation, PF muscle precursors exhibit a moderate delay in differentiation, and PHM muscle precursors show virtually no delay between migration and differentiation. Using lineage tracing, we show that lateral plate contribution to the PHM muscle is minor, unlike its known extensive contribution to the PF muscle and absence in the ventral extension of axial musculature. CONCLUSIONS: We propose that PHM development is intermediate between a migratory muscle mode and an axial muscle mode of development, wherein the PHM differentiates after a very short migration of its precursors and becomes more anterior primarily by elongation of differentiated muscle fibers.

Implementation of the Safe Reduction of Primary Cesarean Births Safety Bundle During the First Year of a Statewide Collaborative in Maryland.

OBJECTIVE: To describe the status of implementation of the Alliance for Innovation in Maternal Health's primary cesarean birth patient safety bundle in Maryland after 1 year (2016-2017), and assess whether hospital characteristics and implementation strategies employed are associated with bundle implementation. METHODS: The Alliance for Innovation in Maternal Health's bundle to decrease primary cesarean births includes 26 evidence-based practices that hospitals can adopt based on specific needs. One year after the start of a statewide implementation collaborative at 31 of 32 birthing hospitals in Maryland, we sent a computer-based survey to hospital collaborative leaders to assess progress. Respondents reported on hospital characteristics, adoption of bundle practices, and use of 15 selected implementation strategies. We conducted descriptive and bivariate analyses of their responses. RESULTS: Among 26 hospitals with complete reporting, 23 fully implemented at least one bundle practice (range 1-7) during the collaborative's first year. Of 26 bundle practices, on average, hospitals had fully implemented a third (mean 8.6; SD 5.5; range 0-17) before the collaborative, and 3 new practices (SD 2.4; range 0-8) during the collaborative. Hospitals' use of six implementation strategies, all highly dependent on strong clinician involvement, was significantly associated with their fully implementing more practices during the collaborative's first year. CONCLUSION: Our assessment has promising results, with a majority of hospitals having implemented new cesarean birth bundle practices during the collaborative's first year. However, there are lessons from the wide variability in the number and type of practices adopted. Clinicians should be aware of this variability and become more involved in the implementation of cesarean birth bundle practices. We identified six strategies associated with full implementation of more bundle practices for which clinicians' support and commitment to practice changes are critical. Clinicians' understanding of available and effective implementation strategies can better assist with the implementation of this and other Alliance for Innovation in Maternal Health patient safety bundles.

Pediatric Program Director Minimum Milestone Expectations Before Allowing Supervision of Others and Unsupervised Practice.

BACKGROUND: The Accreditation Council for Graduate Medical Education requires semiannual Milestone reporting on all residents. Milestone expectations of performance are unknown. OBJECTIVE: To determine pediatric program director (PD) minimum Milestone expectations for residents before being ready to supervise and before being ready to graduate. METHODS: Mixed methods survey of pediatric PDs on their programs' Milestone expectations before residents are ready to supervise and before they are ready to graduate, and in what ways PDs use Milestones to make supervision and graduation decisions. If programs had no established Milestone expectations, PDs indicated expectations they considered for use in their program. Mean minimum Milestone level expectations were adjusted for program size, region, and clustering of Milestone expectations by program were calculated for before supervise and before graduate. Free-text questions were analyzed using thematic analysis. RESULTS: The response rate was 56.8% (113 of 199). Most programs had no required minimum Milestone level before residents are ready to supervise (80%; 76 of 95) or ready to graduate (84%; 80 of 95). For readiness to supervise, minimum Milestone expectations PDs considered establishing for their program were highest for humanism (2.46; 95% confidence interval [CI], 2.21-2.71) and professionalization (2.37; 95% CI, 2.15-2.60). Minimum Milestone expectations for graduates were highest for help-seeking (3.14; 95% CI, 2.83-3.46). Main themes included the use of Milestones in combination with other information to assess learner performance and Milestones are not equally weighted when making advancement decisions. CONCLUSIONS: Most PDs have not established program minimum Milestones, but would vary such expectations according to competency.

Identifying Gaps in the Performance of Pediatric Trainees Who Receive Marginal/Unsatisfactory Ratings.

PURPOSE: To perform a derivation study to determine in which subcompetencies marginal/unsatisfactory pediatric residents had the greatest deficits compared with their satisfactorily performing peers and which subcompetencies best discriminated between marginal/unsatisfactory and satisfactorily performing residents. METHOD: Multi-institutional cohort study of all 21 milestones (rated on four or five levels) reported to the Accreditation Council for Graduate Medical Education, and global marginal/unsatisfactory versus satisfactory performance reported to the American Board of Pediatrics. Data were gathered in 2013-2014. For each level of training (postgraduate year [PGY] 1, 2, and 3), mean differences between milestone levels of residents with marginal/unsatisfactory and satisfactory performance adjusted for clustering by program and C-statistics (area under receiver operating characteristic curve) were calculated. A Bonferroni-corrected significance threshold of .0007963 was used to account for multiple comparisons. RESULTS: Milestone and overall performance evaluations for 1,704 pediatric residents in 41 programs were obtained. For PGY1s, two subcompetencies had almost a one-point difference in milestone levels between marginal/unsatisfactory and satisfactory trainees and outstanding discrimination (≥ 0.90): organize/prioritize (0.93; C-statistic: 0.91) and transfer of care (0.97; C-statistic: 0.90). The largest difference between marginal/unsatisfactory and satisfactory PGY2s was trustworthiness (0.78). The largest differences between marginal/unsatisfactory and satisfactory PGY3s were ethical behavior (1.17), incorporating feedback (1.03), and professionalization (0.96). For PGY2s and PGY3s, no subcompetencies had outstanding discrimination. CONCLUSIONS: Marginal/unsatisfactory pediatric residents had different subcompetency gaps at different training levels. While PGY1s may have global deficits, senior residents may have different performance deficiencies requiring individualized counseling and targeted performance improvement plans.

Competent for Unsupervised Practice: Use of Pediatric Residency Training Milestones to Assess Readiness.

PURPOSE: To describe clinical skills progression during pediatric residency using the distribution of pediatric milestone assessments by subcompetency and year of training and to determine reasonable milestone expectations at time of graduation. METHOD: Multi-institutional cohort study of the milestones reported to the Accreditation Council for Graduate Medical Education for all 21 pediatric subcompetencies. Most subcompetencies were measured using five milestone levels (1 = novice, 2 = advanced beginner, 3 = competent, 4 = proficient, 5 = master); 3 subcompetencies had only four levels defined. RESULTS: Milestone assessments for 2,030 pediatric residents in 47 programs during academic year 2013-2014 were obtained. There was significant variation in end-of-year milestone ratings for residents within each level of training, which decreased as training level increased. Most (78.9%; 434/550) graduating third-year pediatric residents received a milestone rating of ≥ 3 in all 21 subcompetencies; fewer (21.1%; 116/550) received a rating of ≥ 4 in all subcompetencies. Across all training levels, professionalism and interpersonal communication skills were rated highest; quality improvement was rated lowest. CONCLUSIONS: Trainees entered residency with a wide range of skills. As they advanced, skill variability within a training level decreased. Most graduating pediatric residents were still advancing on the milestone continuum toward proficiency and mastery, and an expectation of milestone ratings ≥ 4 in all categories upon graduation is unrealistic; milestone ratings ≥ 3 upon graduation may be more realistic. Understanding current pediatric residents' and graduates' skills can help to identify key areas that should be specifically targeted during training.

The AAP Resilience in the Face of Grief and Loss Curriculum.

A career in pediatrics can bring great joy and satisfaction. It can also be challenging and lead some providers to manifest burnout and depression. A curriculum designed to help pediatric health providers acquire resilience and adaptive skills may be a key element in transforming times of anxiety and grief into rewarding professional experiences. The need for this curriculum was identified by the American Academy of Pediatrics Section on Medical Students, Residents and Fellowship Trainees. A working group of educators developed this curriculum to address the professional attitudes, knowledge, and skills essential to thrive despite the many stressors inevitable in clinical care. Fourteen modules incorporating adult learning theory were developed. The first 2 sections of the curriculum address the knowledge and skills to approach disclosure of life-altering diagnoses, and the second 2 sections focus on the provider's responses to difficult patient care experiences and their needs to develop strategies to maintain their own well-being. This curriculum addresses the intellectual and emotional characteristics patient care medical professionals need to provide high-quality, compassionate care while also addressing active and intentional ways to maintain personal wellness and resilience.

In Pursuit of Meaningful Use of Learning Goals in Residency: A Qualitative Study of Pediatric Residents.

PURPOSE: Medical education aims to equip physicians for lifelong learning, an objective supported by the conceptual framework of self-regulated learning (SRL). Learning goals have been used to develop SRL skills in learners across the medical education continuum. This study's purpose was to elicit residents' perspectives on learning goal use and to develop explanations suggesting how aspects of the learning environment may facilitate or hinder the meaningful use of learning goals in residency. METHOD: Resident focus groups and program director interviews were conducted in 2012-2013, audio-recorded, and transcribed. Programs were selected to maximize diversity of size, geographic location, type of program, and current use of learning goals. Data were analyzed using the constant comparative method associated with grounded theory. Further analysis compared themes frequently occurring together to strengthen the understanding of relationships between the themes. Through iterative discussions, investigators built a grounded theory. RESULTS: Ninety-five third-year residents and 12 program directors at 12 pediatric residency programs participated. The analysis identified 21 subthemes grouped into 5 themes: program support, faculty roles, goal characteristics and purposes, resident attributes, and accountability and goal follow-through. Review of relationships between the themes revealed a pyramid of support with program support as the foundation that facilitates the layers above it, leading to goal follow-through. CONCLUSIONS: Program support facilitates each step of the SRL process that leads to meaningful use of learning goals in residency. A strong foundation of program support should include attention to aspects of the implicit curriculum as well as the explicit curriculum.

Hox5 Genes Regulate the Wnt2/2b-Bmp4-Signaling Axis during Lung Development.

Hox genes are required for proper anteroposterior axial patterning and the development of several organ systems. Here, we show that all three Hox5 paralogous genes play redundant roles in the developing lung. Hoxa5;Hoxb5;Hoxc5 triple-mutant embryos develop severely hypoplastic lungs with reduced branching and proximal-distal patterning defects. Hox5 genes are exclusively expressed in the lung mesoderm; however, defects are observed in both lung mesenchyme and endodermally derived epithelium, demonstrating that Hox5 genes act to regulate mesodermal-epithelial crosstalk during development. We show that Hox5 loss of function leads to loss of Wnt2/2b expression in the distal lung mesenchyme and the downregulation of previously identified downstream targets of Wnt2/2b signaling, including Lef1, Axin2, and Bmp4. Wnt2/2b-enriched media rescue proper Sox2/Sox9 patterning and restore Bmp4 expression in Hox5 triple-mutant lung explants. Taken together, these data show that Hox5 genes are key upstream mesenchymal regulators of the Wnt2/2b-Bmp4-signaling axis critical for proper lung patterning.

Resident Self-Assessment and Learning Goal Development: Evaluation of Resident-Reported Competence and Future Goals.

OBJECTIVE: To determine incidence of learning goals by competency area and to assess which goals fall into competency areas with lower self-assessment scores. METHODS: Cross-sectional analysis of existing deidentified American Academy of Pediatrics' PediaLink individualized learning plan data for the academic year 2009-2010. Residents self-assessed competencies in the 6 Accreditation Council for Graduate Medical Education (ACGME) competency areas and wrote learning goals. Textual responses for goals were mapped to 6 ACGME competency areas, future practice, or personal attributes. Adjusted mean differences and associations were estimated using multiple linear and logistic regression. RESULTS: A total of 2254 residents reported 6078 goals. Residents self-assessed their systems-based practice (51.8) and medical knowledge (53.0) competencies lowest and professionalism (68.9) and interpersonal and communication skills (62.2) highest. Residents were most likely to identify goals involving medical knowledge (70.5%) and patient care (50.5%) and least likely to write goals on systems-based practice (11.0%) and professionalism (6.9%). In logistic regression analysis adjusting for postgraduate year (PGY), gender, and degree type (MD/DO), resident-reported goal area showed no association with the learner's relative self-assessment score for that competency area. In the conditional logistic regression analysis, with each learner serving as his or her own control, senior residents (PGY2/3+s) who rated themselves relatively lower in a competency area were more likely to write a learning goal in that area than were PGY1s. CONCLUSIONS: Senior residents appear to develop better skills and/or motivation to explicitly turn self-assessed learning gaps into learning goals, suggesting that individualized learning plans may help improve self-regulated learning during residency.