Replace, amplify, transform: a qualitative study of how postgraduate trainees and supervisors experience and use telehealth for instruction in ambulatory patient care.

BACKGROUND: Little is known about using telehealth patient visits as an educational mode. Therefore, rapid implementation of telehealth during the COVID-19 pandemic had to be done without understanding how to optimize telehealth for education. With the likely sustained/post-pandemic use of telehealth in ambulatory patient care, filling gaps in our understanding of how telehealth can be used for instruction in this context is critical. This study sought to understand perceptions of pediatric postgraduate trainees and supervisors on the use of telehealth for instruction in ambulatory settings with the goal of identifying effective ways to enhance learning during telehealth visits. METHODS: In May-June of 2020, the authors purposefully sampled first- and third-year postgraduate trainees and supervising attendings from pediatric fellowship programs at one institution that implemented telehealth for instructional activities. They conducted semi-structured interviews; interviews lasted a median of 51 min (trainees) and 41 min (supervisors). They conducted interviews and data analysis iteratively until reaching saturation. Using thematic analysis, they created codes and constructed themes from coded data. They organized themes using the Replace-Amplify-Transform (RAT) model, which proposes that technology can replace in-person learning and/or amplify and transform learning. RESULTS: First-year trainees (n = 6), third-year trainees (n = 5) and supervisors (n = 6) initially used telehealth to replace in-person learning. However, skills that could be practiced in telehealth visits differed from in-person visits and instructional activities felt rushed or awkward. Trainees and supervisors adapted and used telehealth to amplify learning by enhancing observation and autonomy. They also transformed learning, using telehealth to develop novel skills. CONCLUSIONS: To harness telehealth for instructional activities, our findings indicated that trainees and supervisors should shift from using it as a direct replacement for in-person education to taking advantage of novel opportunities to amplify and transform education in PGME. The authors provide data-driven recommendations to help PGME trainees, supervisors and educators capitalize on the educational advantages of telehealth.

"All the ward's a stage": a qualitative study of the experience of direct observation of handoffs.

Direct observation of clinical skills is central to assessment in a competency-based medical education model, yet little is known about how direct observation is experienced by trainees and observers. The objective of the study is to explore how direct observation was experienced by residents and faculty in the context of the I-PASS Handoff Study. In this multi-center qualitative study, we conducted focus groups and semi-structured interviews of residents and faculty members at eight tertiary pediatric centers in North America that implemented the I-PASS Handoff Bundle. We employed qualitative thematic analysis to interpret the data. Barriers to and strategies for direct observation were described relating to the observer, trainee, and clinical environment. Residents and faculty described a mutual awareness that residents change their performance of handoffs when observed, in contrast to their usual behavior in a clinical setting. Changes in handoff performance may depend on the nature of the observer or 'audience'. Direct observation also highlighted the importance of handoffs to participants, recognized as a clinical activity that warrants feedback and assessment. Dramaturgical theory can be used to understand our finding of 'front-stage' (observed) versus 'backstage' (unobserved) handoffs as distinct performances, tailored to an "audience". Educators must be cognizant of changes in performance of routine clinical activities when using direct observation to assess clinical competence.

Developing Standardized "Receiver-Driven" Handoffs Between Referring Providers and the Emergency Department: Results of a Multidisciplinary Needs Assessment.

BACKGROUND: Miscommunication during patient transfers is a leading cause of medical errors. Inpatient standardization of handoff communication has been associated with reduced medical errors, but less is known about best practices for handoffs from referring providers to the emergency department (ED). The study aims were to identify (1) stakeholder perceptions of current handoff processes and (2) key handoff elements and strategies to optimize patient care on transfer. METHODS: A mixed-methods needs assessment study was conducted at a tertiary care children's hospital with a communication center that receives verbal handoff via telephone from referring providers and provides written summary to the ED. ED, primary care providers, and communication center staff were surveyed to understand perceptions of handoff processes and ideal handoff elements. Focus groups were conducted to refine concepts. Descriptive statistics, chi-square analysis, and qualitative content analysis were used to analyze responses. RESULTS: The survey response rate was 129/152 providers (85%). Forty-two percent of respondents described the quality of the handoff process as "very good" or "excellent"; 43% reported miscommunication occurring "sometimes" or "frequently." Within the I-PASS framework-Illness severity, Patient summary, Action list, Situation awareness and contingency planning, and Synthesis by receiver-respondents identified 10 key elements to obtain through a receiver-driven process to optimize care on transfer. Free-text responses revealed a perceived need to standardize communication. CONCLUSION: A minority of providers perceived handoff quality between outpatient practices and the ED as "very good" or "excellent"; almost half perceived regular miscommunication. A receiver-driven process is a novel approach that may help ensure standardized communication of key handoff elements in this context.

Changes in medical errors after implementation of a handoff program.

BACKGROUND: Miscommunications are a leading cause of serious medical errors. Data from multicenter studies assessing programs designed to improve handoff of information about patient care are lacking. METHODS: We conducted a prospective intervention study of a resident handoff-improvement program in nine hospitals, measuring rates of medical errors, preventable adverse events, and miscommunications, as well as resident workflow. The intervention included a mnemonic to standardize oral and written handoffs, handoff and communication training, a faculty development and observation program, and a sustainability campaign. Error rates were measured through active surveillance. Handoffs were assessed by means of evaluation of printed handoff documents and audio recordings. Workflow was assessed through time-motion observations. The primary outcome had two components: medical errors and preventable adverse events. RESULTS: In 10,740 patient admissions, the medical-error rate decreased by 23% from the preintervention period to the postintervention period (24.5 vs. 18.8 per 100 admissions, P<0.001), and the rate of preventable adverse events decreased by 30% (4.7 vs. 3.3 events per 100 admissions, P<0.001). The rate of nonpreventable adverse events did not change significantly (3.0 and 2.8 events per 100 admissions, P=0.79). Site-level analyses showed significant error reductions at six of nine sites. Across sites, significant increases were observed in the inclusion of all prespecified key elements in written documents and oral communication during handoff (nine written and five oral elements; P<0.001 for all 14 comparisons). There were no significant changes from the preintervention period to the postintervention period in the duration of oral handoffs (2.4 and 2.5 minutes per patient, respectively; P=0.55) or in resident workflow, including patient-family contact and computer time. CONCLUSIONS: Implementation of the handoff program was associated with reductions in medical errors and in preventable adverse events and with improvements in communication, without a negative effect on workflow. (Funded by the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, and others.).

Integrating Research, Quality Improvement, and Medical Education for Better Handoffs and Safer Care: Disseminating, Adapting, and Implementing the I-PASS Program.

BACKGROUND: In 2009 the I-PASS Study Group was formed by patient safety, medical education, health services research, and clinical experts from multiple institutions in the United States and Canada. When the I-PASS Handoff Program, which was developed by the I-PASS Study Group, was implemented in nine hospitals, it was associated with a 30% reduction in injuries due to medical errors and significant improvements in handoff processes, without any adverse effects on provider work flow. METHODS: To effectively disseminate and adapt I-PASS for use across specialties and disciplines, a series of federally and privately funded dissemination and implementation projects were carried out following the publication of the initial study. The results of these efforts have informed ongoing initiatives intended to continue adapting and scaling the program. RESULTS: As of this writing, I-PASS Study Group members have directly worked with more than 50 hospitals to facilitate implementation of I-PASS. To further disseminate I-PASS, Study Group members delivered hundreds of academic presentations, including plenaries at scientific meetings, workshops, and institutional Grand Rounds. Some 3,563 individuals, representing more than 500 institutions in the 50 states in the United States, the District of Columbia, Puerto Rico, and 57 other countries, have requested access to I-PASS materials. Most recently, the I-PASSSM Patient Safety Institute has developed a virtual immersion training platform, mobile handoff observational tools, and processes to facilitate further spread of I-PASS. CONCLUSION: Implementation of I-PASS has been associated with substantial improvements in patient safety and can be applied to a variety of disciplines and types of patient handoffs. Widespread implementation of I-PASS has the potential to substantially improve patient safety in the United States and beyond.

Conditional Survival and Predictors of Late Death in Patients With Ewing Sarcoma.

PURPOSE: Long-term survivors of Ewing sarcoma (EWS) are at considerable risk for future complications, including late relapse and death. Data on prognostic factors for late death in those who have survived beyond 5 years are lacking. METHODS: We conducted a retrospective cohort study using the Surveillance, Epidemiology, and End Results database. We obtained clinical features and outcome data on 1,351 patients with EWS who had survived 60 months or more. From these data, we performed univariate and multivariable analyses of overall survival (OS) using log-rank tests and Cox proportional hazard models. RESULTS: Of 1,351 patients in the cohort, there were 209 deaths, 144 (69%) of which were reported to be due to EWS. The OS for 5-year survivors at 10 years was 87.5% (95% confidence interval 85.4-89.3%). Univariate adverse prognostic factors for late death in 5-year survivors included age ≥ 18 years at initial diagnosis, male sex, and axial/pelvic primary site. Initial stage was not prognostic. Independent adverse prognostic factors for late death included black race (hazard ratio [HR] 2.16, P = 0.01), age ≥ 18 years at diagnosis (HR 2.02, P < 0.001), male sex (HR 1.43, P = 0.01), and axial/pelvic primary site (HR 1.43, P = 0.02). CONCLUSIONS: The majority of late deaths in 5-year survivors are due to EWS. Black race, age ≥18 at diagnosis, male sex, and axial/pelvic primary site (but not stage at diagnosis) are independently associated with increased risk of late death.

Implementing a Family-Centered Rounds Intervention Using Novel Mentor-Trios.

BACKGROUND AND OBJECTIVES: Patient and Family Centered I-PASS (PFC I-PASS) emphasizes family and nurse engagement, health literacy, and structured communication on family-centered rounds organized around the I-PASS framework (Illness severity-Patient summary-Action items-Situational awareness-Synthesis by receiver). We assessed adherence, safety, and experience after implementing PFC I-PASS using a novel "Mentor-Trio" implementation approach with multidisciplinary parent-nurse-physician teams coaching sites. METHODS: Hybrid Type II effectiveness-implementation study from 2/29/19-3/13/22 with ≥3 months of baseline and 12 months of postimplementation data collection/site across 21 US community and tertiary pediatric teaching hospitals. We conducted rounds observations and surveyed nurses, physicians, and Arabic/Chinese/English/Spanish-speaking patients/parents. RESULTS: We conducted 4557 rounds observations and received 2285 patient/family, 1240 resident, 819 nurse, and 378 attending surveys. Adherence to all I-PASS components, bedside rounding, written rounds summaries, family and nurse engagement, and plain language improved post-implementation (13.0%-60.8% absolute increase by item), all P < .05. Except for written summary, improvements sustained 12 months post-implementation. Resident-reported harms/1000-resident-days were unchanged overall but decreased in larger hospitals (116.9 to 86.3 to 72.3 pre versus early- versus late-implementation, P = .006), hospitals with greater nurse engagement on rounds (110.6 to 73.3 to 65.3, P < .001), and greater adherence to I-PASS structure (95.3 to 73.6 to 72.3, P < .05). Twelve of 12 measures of staff safety climate improved (eg, "excellent"/"very good" safety grade improved from 80.4% to 86.3% to 88.0%), all P < .05. Patient/family experience and teaching were unchanged. CONCLUSIONS: Hospitals successfully used Mentor-Trios to implement PFC I-PASS. Family/nurse engagement, safety climate, and harms improved in larger hospitals and hospitals with better nurse engagement and intervention adherence. Patient/family experience and teaching were not affected.

Program Evaluation in Health Professions Education: An Innovative Approach Guided by Principles.

PROBLEM: Program evaluation approaches that center the achievement of specific, measurable, achievable, realistic, and time-bound goals are common in health professions education (HPE) but can be challenging to articulate when evaluating emergent programs. Principles-focused evaluation is an alternative approach to program evaluation that centers on adherence to guiding principles, not achievement of goals. The authors describe their innovative application of principles-focused evaluation to an emergent HPE program. APPROACH: The authors applied principles-focused evaluation to the Children's Hospital of Philadelphia Medical Education Collaboratory, a works-in-progress program for HPE scholarship. In September 2019, the authors drafted 3 guiding principles. In May 2021, they used feedback from Collaboratory attendees to revise the guiding principles: Advance Excellence , Build Bridges , and Cultivate Learning . OUTCOMES: In July 2021, the authors queried participants about the extent to which their experience with the Collaboratory adhered to the revised guiding principles. Twenty of the 38 Collaboratory participants (53%) responded to the survey. Regarding the guiding principle Advance Excellence , 9 respondents (45%) reported that the Collaboratory facilitated engagement in scholarly conversation only by a small extent, and 8 (40%) reported it facilitated professional growth only by a small extent. Although some respondents expressed positive regard for the high degree of rigor promoted by the Collaboratory, others felt discouraged because this degree of rigor seemed unachievable. Regarding the guiding principle Build Bridges , 19 (95%) reported the Collaboratory welcomed perspectives within the group. Regarding the guiding principle Cultivate Learning , 19 (95%) indicated the Collaboratory welcomed perspectives within the group and across disciplines, and garnered collaboration. NEXT STEPS: Next steps include improving adherence to the principle of Advancing Excellence , fostering a shared mental model of the Collaboratory's guiding principles, and applying a principles-focused approach to the evaluation of multi-site HPE programs.

Measuring Training Disruptions Using an Informatics Based Tool.

OBJECTIVE: Training disruptions, such as planned curricular adjustments or unplanned global pandemics, impact residency training in ways that are difficult to quantify. Informatics-based medical education tools can help measure these impacts. We tested the ability of a software platform driven by electronic health record data to quantify anticipated changes in trainee clinical experiences during the COVID-19 pandemic. METHODS: We previously developed and validated the Trainee Individualized Learning System (TRAILS) to identify pediatric resident clinical experiences (i.e. shifts, resident provider-patient interactions (rPPIs), and diagnoses). We used TRAILS to perform a year-over-year analysis comparing pediatrics residents at a large academic children's hospital during March 15-June 15 in 2018 (Control #1), 2019 (Control #2), and 2020 (Exposure). RESULTS: Residents in the exposure cohort had fewer shifts than those in both control cohorts (P < .05). rPPIs decreased an average of 43% across all PGY levels, with interns experiencing a 78% decrease in Continuity Clinic. Patient continuity decreased from 23% to 11%. rPPIs with common clinic and emergency department diagnoses decreased substantially during the exposure period. CONCLUSIONS: Informatics tools like TRAILS may help program directors understand the impact of training disruptions on resident clinical experiences and target interventions to learners' needs and development.

Implementation of the I-PASS handoff program in diverse clinical environments: A multicenter prospective effectiveness implementation study.

BACKGROUND: Handoff miscommunications are a leading source of medical errors. Harmful medical errors decreased in pediatric academic hospitals following implementation of the I-PASS handoff improvement program. However, implementation across specialties has not been assessed. OBJECTIVE: To determine if I-PASS implementation across diverse settings would be associated with improvements in patient safety and communication. DESIGN: Prospective Type 2 Hybrid effectiveness implementation study. SETTINGS AND PARTICIPANTS: Residents from diverse specialties across 32 hospitals (12 community, 20 academic). INTERVENTION: External teams provided longitudinal coaching over 18 months to facilitate implementation of an enhanced I-PASS program and monthly metric reviews. MAIN OUTCOME AND MEASURES: Systematic surveillance surveys assessed rates of resident-reported adverse events. Validated direct observation tools measured verbal and written handoff quality. RESULTS: 2735 resident physicians and 760 faculty champions from multiple specialties (16 internal medicine, 13 pediatric, 3 other) participated. 1942 error surveillance reports were collected. Major and minor handoff-related reported adverse events decreased 47% following implementation, from 1.7 to 0.9 major events/person-year (p < .05) and 17.5 to 9.3 minor events/person-year (p < .001). Implementation was associated with increased inclusion of all five key handoff data elements in verbal (20% vs. 66%, p < .001, n = 4812) and written (10% vs. 74%, p < .001, n = 1787) handoffs, as well as increased frequency of handoffs with high quality verbal (39% vs. 81% p < .001) and written (29% vs. 78%, p < .001) patient summaries, verbal (29% vs. 78%, p < .001) and written (24% vs. 73%, p < .001) contingency plans, and verbal receiver syntheses (31% vs. 83%, p < .001). Improvement was similar across provider types (adult vs. pediatric) and settings (community vs. academic).

Association of Patient and Family Reports of Hospital Safety Climate With Language Proficiency in the US.

Importance: Patients with language barriers have a higher risk of experiencing hospital safety events. This study hypothesized that language barriers would be associated with poorer perceptions of hospital safety climate relating to communication openness. Objective: To examine disparities in reported hospital safety climate by language proficiency in a cohort of hospitalized children and their families. Design, Setting, and Participants: This cohort study conducted from April 29, 2019, through March 1, 2020, included pediatric patients and parents or caregivers of hospitalized children at general and subspecialty units at 21 US hospitals. Randomly selected Arabic-, Chinese-, English-, and Spanish-speaking hospitalized patients and families were approached before hospital discharge and were included in the analysis if they provided both language proficiency and health literacy data. Participants self-rated language proficiency via surveys. Limited English proficiency was defined as an answer of anything other than "very well" to the question "how well do you speak English?" Main Outcomes and Measures: Primary outcomes were top-box (top most; eg, strongly agree) 5-point Likert scale ratings for 3 Children's Hospital Safety Climate Questionnaire communication openness items: (1) freely speaking up if you see something that may negatively affect care (top-box response: strongly agree), (2) questioning decisions or actions of health care providers (top-box response: strongly agree), and (3) being afraid to ask questions when something does not seem right (top-box response: strongly disagree [reverse-coded item]). Covariates included health literacy and sociodemographic characteristics. Logistic regression was used with generalized estimating equations to control for clustering by site to model associations between openness items and language proficiency, adjusting for health literacy and sociodemographic characteristics. Results: Of 813 patients, parents, and caregivers who were approached to participate in the study, 608 completed surveys (74.8% response rate). A total of 87.7% (533 of 608) of participants (434 [82.0%] female individuals) completed language proficiency and health literacy items and were included in the analyses; of these, 14.1% (75) had limited English proficiency. Participants with limited English proficiency had lower odds of freely speaking up if they see something that may negatively affect care (adjusted odds ratio [aOR], 0.26; 95% CI, 0.15-0.43), questioning decisions or actions of health care providers (aOR, 0.19; 95% CI, 0.09-0.41), and being unafraid to ask questions when something does not seem right (aOR, 0.44; 95% CI, 0.27-0.71). Individuals with limited health literacy (aOR, 0.66; 95% CI, 0.48-0.91) and a lower level of educational attainment (aOR, 0.59; 95% CI, 0.36-0.95) were also less likely to question decisions or actions. Conclusions and Relevance: This cohort study found that limited English proficiency was associated with lower odds of speaking up, questioning decisions or actions of providers, and being unafraid to ask questions when something does not seem right. This disparity may contribute to higher hospital safety risk for patients with limited English proficiency. Dedicated efforts to improve communication with patients and families with limited English proficiency are necessary to improve hospital safety and reduce disparities.

Clinical features and outcomes in patients with extraskeletal Ewing sarcoma.

BACKGROUND: Ewing sarcoma can arise in either bone or soft tissue. The purpose of this study was to investigate whether patient characteristics, treatment strategies, and outcomes differ between skeletal Ewing sarcoma and extraskeletal Ewing sarcoma (EES). METHODS: Patients <40 years of age with Ewing sarcoma or peripheral primitive neuroectodermal tumor reported to the United States Surveillance, Epidemiology, and End Results Program database from 1973 to 2007 were evaluated based on skeletal (n = 1519) versus extraskeletal (n = 683) site of origin. Patient characteristics were compared using Fisher exact tests. Overall survival was estimated via the Kaplan-Meier method and compared using log-rank tests and Cox proportional hazard models. RESULTS: Patients with EES had a higher mean age (19.5 vs 16.3 years; P < .001) and were less likely to be male (53.4% vs 63.3%; P < .001) or white (84.8% vs 92.5%; P < .001) compared with patients with skeletal tumors. Extraskeletal tumors were more likely to arise in axial locations (72.9% vs 54.2%; P = .001) but were less likely to arise specifically in the pelvis (19.8% vs 26.6%; P < .001). Metastatic status or tumor size did not differ by group. Five-year overall survival was superior for localized EES compared with localized skeletal tumors (69.7% vs 62.6%; P = .02). The hazard ratio for death in patients with localized skeletal tumors compared with localized EES was 2.36 (95% confidence interval, 1.61-3.44) beyond 24 months from initial diagnosis. CONCLUSIONS: Patient characteristics and outcomes differ among patients with EES compared with patients with skeletal Ewing sarcoma. These findings may have important implications for patient care.

Investigation of the insulin-like growth factor-1 signaling pathway in localized Ewing sarcoma: a report from the Children's Oncology Group.

BACKGROUND: The insulin-like growth factor-1 (IGF-1) signaling pathway plays an important role in the pathology of Ewing sarcoma (ES). Retrospective studies have suggested that levels of IGF-1 and IGF binding protein 3 (IGFBP-3) are correlated with the outcome of patients with ES. METHODS: The IGF-1 signaling pathway was investigated prospectively in 269 patients who had localized, previously untreated ES. Serum samples were obtained at diagnosis, and concentrations of IGF-1 and IGFBP-3 were determined by enzyme-linked immunosorbent assays. In addition, immunohistochemistry (IHC) was performed to assay for phosphorylated p70S6 kinase, protein kinase B (Akt), and forkhead box protein O1 (FOXO1) and to determine the presence of protein tyrosine phosphatase-L1 (PTPL1). IHC findings along with IGF-1 and IGFBP-3 concentrations were correlated with age, tumor location, sex, event-free survival, and overall survival. RESULTS: Patients aged >18 years tended to have higher levels of IGF-1 (P = .10), lower levels of IGFBP-3 (P = .16), and decreased IGFBP-3:IGF-1 ratios (P = .01). No correlations were observed between sex, tumor location, or outcomes and concentrations of IGF-1 or IGFBP-3. Phosphorylation of p70S6 kinase, Akt, and FOXO1 was detected in the majority of patient tissues but was not associated with age, sex, or tumor location. PTPL1 was present in >80% of tumors and also was not correlated with age, sex, or tumor location. There was no difference in survival with respect to the presence of phosphorylated p70S6 kinase, phosphorylated FOXO1, phosphorylated Akt, or PTPL1. CONCLUSIONS: The baseline IGFBP-3:IGF-1 ratio was correlated with age but did not affect the outcomes of patients with ES. The authors concluded that additional investigation of the IGF-1 pathway is warranted in patients with ES, and especially in those who have received treatment with IGF-1 receptor antibody inhibitors.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review.

PURPOSE: Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions. METHOD: The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders. RESULTS: In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain. CONCLUSIONS: Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Entrustable Professional Activities and Entrustment Decision Making: A Development and Research Agenda for the Next Decade.

To establish a research and development agenda for Entrustable Professional Activities (EPAs) for the coming decade, the authors, all active in this area of investigation, reviewed recent research papers, seeking recommendations for future research. They pooled their knowledge and experience to identify 3 levels of potential research and development: the micro level of learning and teaching; the meso level of institutions, programs, and specialty domains; and the macro level of regional, national, and international dynamics. Within these levels, the authors categorized their recommendations for research and development. The authors identified 14 discrete themes, each including multiple questions or issues for potential exploration, that range from foundational and conceptual to practical. Much research to date has focused on a variety of issues regarding development and early implementation of EPAs. Future research should focus on large-scale implementation of EPAs to support competency-based medical education (CBME) and on its consequences at the 3 levels. In addition, emerging from the implementation phase, the authors call for rigorous studies focusing on conceptual issues. These issues include the nature of entrustment decisions and their relationship with education and learner progress and the use of EPAs across boundaries of training phases, disciplines and professions, including continuing professional development. International studies evaluating the value of EPAs across countries are another important consideration. Future studies should also remain alert for unintended consequences of the use of EPAs. EPAs were conceptualized to support CBME in its endeavor to improve outcomes of education and patient care, prompting creation of this agenda.

Implementing receiver-driven handoffs to the emergency department to reduce miscommunication.

BACKGROUND: Miscommunications during care transfers are a leading cause of medical errors. Recent consensus-based recommendations to standardise information transfer from outpatient clinics to the emergency department (ED) have not been formally evaluated. We sought to determine whether a receiver-driven structured handoff intervention is associated with 1) increased inclusion of standardised elements; 2) reduced miscommunications and 3) increased perceived quality, safety and efficiency. METHODS: We conducted a prospective intervention study in a paediatric ED and affiliated clinics in 2016-2018. We developed a bundled handoff intervention included a standard template, receiver training, awareness campaign and iterative feedback. We assessed a random sample of audio-recorded handoffs and associated medical records to measure rates of inclusion of standardised elements and rate of miscommunications. We surveyed key stakeholders pre-intervention and post-intervention to assess perceptions of quality, safety and efficiency of the handoff process. RESULTS: Across 162 handoffs, implementation of a receiver-driven intervention was associated with significantly increased inclusion of important elements, including illness severity (46% vs 77%), tasks completed (64% vs 83%), expectations (61% vs 76%), pending tests (0% vs 64%), contingency plans (0% vs 54%), detailed callback request (7% vs 81%) and synthesis (2% vs 73%). Miscommunications decreased from 48% to 26%, a relative reduction of 23% (95% CI -39% to -7%). Perceptions of quality (35% vs 59%), safety (43% vs 73%) and efficiency (17% vs 72%) improved significantly post-intervention. CONCLUSIONS: Implementation of a receiver-driven intervention to standardise clinic-to-ED handoffs was associated with improved communication quality. These findings suggest that expanded implementation of similar programmes may significantly improve the care of patients transferred to the paediatric ED.

Learner Levels of Supervision Across the Continuum of Pediatrics Training.

PURPOSE: To describe trajectories in level of supervision ratings for linked entrustable professional activities (EPAs) among pediatric learners in medical school, residency, fellowship. METHOD: The authors performed secondary analyses of 3 linked datasets of level of supervision ratings for the Core EPAs for Entering Residency, the General Pediatrics EPAs, and the Subspecialty Pediatrics EPAs. After identifying 9 activities in common across training stages and aligning the level of entrustment-supervision scales across the datasets, piecewise ordinal and linear mixed effects models were fitted to characterize trajectories of supervision ratings. RESULTS: Within each training period, learners were rated as needing less supervision over time in each activity. When transitioning from medical school to residency or during the first year of residency, learners were rated as needing greater supervision in activities related to patient management, teamwork, emergent care, and public health/QI than in earlier periods. When transitioning from residency to fellowship, learners were always rated as needing greater supervision than they had been accorded at the end of residency and sometimes even more than they had been accorded at the start of residency. CONCLUSIONS: Although development over training is often imagined as continuous and monotonically increasing competence, this study provides empirical evidence supporting the idea that entrustment is a set of discrete decisions. The relaxation of supervision in training is not a linear process. Even with a seamless curriculum, supervision is tightly bound to the training setting. Several explanations for these findings are discussed.

I-PASS Mentored Implementation Handoff Curriculum: Frontline Provider Training Materials.

Introduction: The I-PASS Handoff Program is a comprehensive handoff curriculum that has been shown to decrease rates of medical errors and adverse events during patient handoffs. Frontline providers are the key individuals participating in handoffs of patient care. It is important they receive robust handoff training. Methods: The I-PASS Mentored Implementation Handoff Curriculum frontline provider training materials were created as part of the original I-PASS Study and adapted for the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program. The adapted materials embrace a flipped classroom approach with an emphasis on adult learning theory principles. The training includes an overview of I-PASS handoff techniques, TeamSTEPPS team communication strategies, verbal handoff simulation scenarios, and a printed handoff document exercise. Results: As part of the SHM I-PASS Mentored Implementation Program, 2,735 frontline providers were trained at 32 study sites (16 adult and 16 pediatric) across North America. At the end of their training, 1,762 frontline providers completed the workshop evaluation form (64% response rate). After receiving the training, over 90% agreed/strongly agreed that they were able to distinguish a good- from a poor-quality handoff, articulate the elements of the I-PASS mnemonic, construct a high-quality patient summary, advocate for an appropriate environment for handoffs, and participate in handoff simulations. Universally, the training provided them with knowledge and skills relevant to their patient care activities. Discussion: The I-PASS frontline training materials were rated highly by those trained and are an integral part of a successful I-PASS Handoff Program implementation.

Shared Mental Models Among Clinical Competency Committees in the Context of Time-Variable, Competency-Based Advancement to Residency.

PURPOSE: To evaluate response process validity evidence for clinical competency committee (CCC) assessments of first-year residents on a subset of General Pediatrics Entrustable Professional Activities (EPAs) and milestones in the context of a national pilot of competency-based, time-variable (CBTV) advancement from undergraduate to graduate medical education. METHOD: Assessments of 2 EPAs and 8 milestones made by the trainees' actual CCCs and 2 different blinded "virtual" CCCs for 48 first-year pediatrics residents at 4 residency programs between 2016 and 2018 were compared. Residents had 3 different training paths from medical school to residency: time-variable graduation at the same institution as their residency, time-fixed graduation at the same institution, or time-fixed graduation from a different institution. Assessments were compared using ordinal mixed-effects models. RESULTS: Actual CCCs assigned residents higher scores than virtual CCCs on milestones and one EPA's supervision levels. Residents who graduated from a different institution than their residency received lower milestone ratings than either group from the same institution; CBTV residents received higher ratings on one milestone (ICS4) and similar ratings on all others compared with non-CBTV residents who completed medical school at the same institution. CONCLUSIONS: First-year residents who graduated from CBTV medical school programs were assessed as having the same level of competence as residents who graduated from traditional medical school programs, but response process evidence suggests that members of CCCs may also draw on undocumented personal knowledge of the learner to draw conclusions about resident competence.

Longitudinal Assessment of Resident Performance Using Entrustable Professional Activities.

Importance: Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. Objective: To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. Design, Setting, and Participants: Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. Interventions: Longitudinal, prospective assessment using EPAs. Main Outcomes and Measures: Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. Results: Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). Conclusions and Relevance: This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence.