Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

BACKGROUND AND OBJECTIVES: Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS: We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS: The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS: Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

The burden of atopic dermatitis in Australia: Medical care, treatment satisfaction, financial impact, long-term control and quality of life.

Atopic dermatitis patients and caregivers experience a high physical, mental and financial burden in Australia. We outline how the current care model impacts disease management and patients' quality of life via a survey of 265 Australian patients and caregivers to capture the experience of managing atopic dermatitis (AD) in Australia. Patients report an unsatisfactory quality of life and a high burden of disease with poor long-term control and low treatment satisfaction. They also reported changing spending and saving patterns to fund medical care. Patient experience improves with more specialised care that incorporates shared decision-making and patient/caregiver training beyond the clinic visit. These results highlight the need for improved general practitioner (GP) education, expedited access to medical specialists, greater patient involvement in treatment choices and more financial assistance to improve the experience of Australian patients and caregivers.

Patients' and Caregivers' Experiences Navigating the Burden of Atopic Dermatitis in Argentina.

Background and Objectives: Little is known about patients' and caregivers' experiences with atopic dermatitis (AD) in Argentina, so a survey was administered to learn more. Materials and Methods: A 53-item anonymous survey was administered in Spanish to adult AD patients (n = 334) and caregivers (n = 339) of pediatric AD patients in Argentina (total n = 673). Demographics, healthcare provider information, financial burden, disease severity, disease burden, level of disease-specific education, and experience with shared physician/patient decision making were collected. Linear and logistic regression models were used for statistical comparisons. Results: Survey respondents were overwhelmingly female (90.8%), as was the overall patient population (72.8%). Patients were seen mostly by healthcare specialists (66.8% dermatologists, 13.5% pediatricians, 7.7% allergists, and 7.2% general practitioners). Only 2.8% of respondents reported no symptoms, while 33.3%, 52.4%, and 11.5% reported mild, moderate, and severe AD disease, respectively. Anxiety/depression and pain/discomfort were the most impactful on respondents' quality of life. Caregivers of children with moderate to severe AD and adult patients with severe AD reported a significant financial burden, including using savings or not purchasing food or other essentials to afford medical care. Few people reported receiving disease-specific education or having their own treatment priorities taken into consideration. For adult patients, receiving disease education and being asked about treatment priorities were associated with higher treatment satisfaction and AD control. Discussion: Mental health, pain/discomfort, and financial worries are the most important burdens for adult AD patients and caregivers of children with AD in Argentina. We recommend prioritizing disease-specific education and shared decision making to improve AD care in Argentina.

Association between sleep disorders and health care utilization in children with chronic medical conditions: a Medicaid claims data analysis.

STUDY OBJECTIVES: To examine the risk of increased health care utilization (HU) linked to individual sleep disorders in children with chronic medical conditions. METHODS: Medicaid claims data from a cohort of 16,325 children enrolled in the Coordinated Healthcare for Complex Kids (CHECK) project were used. Sleep disorders and chronic medical conditions were identified using International Classification of Diseases, Ninth, and 10th Revision, codes. Three HU groups were identified based on participants' prior hospitalizations and emergency department (ED) visits in the 12 months prior to enrollment: low (no hospitalization or ED visit), medium (1-2 hospitalizations or 1-3 ED visits), and high (≥ 3 hospitalizations or ≥ 4 ED visits). The odds of being in an increased HU group associated with specific sleep disorders after controlling for confounding factors were examined. RESULTS: Children with chronic medical conditions and any sleep disorder had nearly twice the odds (odds ratio = 1.83; 95% confidence interval: 1.67-2.01) of being in an increased HU group compared with those without a sleep disorder. The odds of being in the increased HU group varied among sleep disorders. Only sleep-disordered breathing (odds ratio = 1.51; 95% confidence interval : 1.17-1.95), insomnia (odds ratio = 1.46; 95% confidence interval : 1.06-2.02), and circadian rhythm sleep disorder (odds ratio = 2.45; 95% confidence interval : 1.07-5.64) increased those odds. Younger age and being White were also linked to increased HU. CONCLUSIONS: Sleep disorders are associated with increased risk of heightened HU (ED visits and/or hospitalizations) in children with chronic medical conditions. This risk varies by specific sleep disorders. These findings indicate the need for careful evaluation and management of sleep disorders in this high-risk cohort. CITATION: Adavadkar PA, Brooks L, Pappalardo AA, Schwartz A, Rasinski K, Martin MA. Association between sleep disorders and health care utilization in children with chronic medical conditions: a Medicaid claims data analysis. J Clin Sleep Med. 2024;20(4):595-601.

Developing Evidence for Equitable Assessment Characteristics Based on Clinical Learner Preferences Using Discrete Choice Experiments.

PURPOSE: Medical education is only beginning to explore the factors that contribute to equitable assessment in clinical settings. Increasing knowledge about equitable assessment ensures a quality medical education experience that produces an excellent, diverse physician workforce equipped to address the health care disparities facing patients and communities. Through the lens of the Anti-Deficit Achievement framework, the authors aimed to obtain evidence for a model for equitable assessment in clinical training. METHOD: A discrete choice experiment approach was used which included an instrument with 6 attributes each at 2 levels to reveal learner preferences for the inclusion of each attribute in equitable assessment. Self-identified underrepresented in medicine (UIM) and not underrepresented in medicine (non-UIM) (N = 306) fourth-year medical students and senior residents in medicine, pediatrics, and surgery at 9 institutions across the United States completed the instrument. A mixed-effects logit model was used to determine attributes learners valued most. RESULTS: Participants valued the inclusion of all assessment attributes provided except for peer comparison. The most valued attribute of an equitable assessment was how learner identity, background, and trajectory were appreciated by clinical supervisors. The next most valued attributes were assessment of growth, supervisor bias training, narrative assessments, and assessment of learner's patient care, with participants willing to trade off any of the attributes to get several others. There were no significant differences in value placed on assessment attributes between UIM and non-UIM learners. Residents valued clinical supervisors valuing learner identity, background, and trajectory and clinical supervisor bias training more so than medical students. CONCLUSIONS: This study offers support for the components of an antideficit-focused model for equity in assessment and informs efforts to promote UIM learner success and guide equity, diversity, and inclusion initiatives in medical education.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment.

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Upholding Our PROMISE: Underrepresented in Medicine Pediatric Residents' Perspectives on Interventions to Promote Belonging.

PURPOSE: Underrepresented in medicine (UIM) residents experience challenges during training that threaten their sense of belonging in medicine; therefore, residency programs should intentionally implement interventions to promote belonging. This study explored UIM pediatric residents' perspectives on current residency program measures designed to achieve this goal. METHOD: The authors conducted a secondary qualitative analysis as part of a national cross-sectional study, PROmoting Med-ed Insight into Supportive Environments (PROMISE), which explored pediatric residents' experiences and perspectives during training in relation to their self-identities. A 23-item web-based survey was distributed through the Association of Pediatric Program Directors Longitudinal Educational Research Assessment Network from October 2020 to January 2021. Participants provided free-text responses to the question "What are current measures that promote a sense of belonging for the UIM community in your training program?" The authors used conventional content analysis to code and identify themes in responses from UIM participants. RESULTS: Of the 1,748 residents invited to participate, 931 (53%) residents from 29 programs completed the survey, with 167 (18%) identifying as UIM. Of the 167 UIM residents, 74 (44%) residents from 22 programs responded to the free-text question. The authors coded more than 140 unique free-text responses and identified 7 major themes: (1) critical mass of UIM residents; (2) focused recruitment of UIM residents; (3) social support, including opportunities to build community among UIM residents; (4) mentorship; (5) caring and responsive leadership; (6) education on health disparities, diversity, equity, inclusion, and antiracism; and (7) opportunities to serve, including giving back to the local community and near-peer mentorship of UIM premedical and medical students. CONCLUSIONS: This is the first national study to describe UIM pediatric residents' perspectives on interventions that promote a sense of belonging. Programs should consider implementing these interventions to foster inclusion and belonging among UIM trainees.

Personalized Postacute Hospitalization Recovery: A Novel Intervention to Improve Patient Experience and Reduce Cost.

GOAL: Readmissions are a significant financial burden for payers. Cardiovascular-related discharges are particularly prone to readmission. Posthospital discharge support can impact patient recovery and probably reduce patient readmissions. This study aimed to address the underlying behavioral and psychosocial factors that can negatively affect patients after discharge. METHODS: The study population was adult patients admitted to the hospital with a cardiovascular diagnosis who had a plan to discharge home. Those who consented to participate were randomized to intervention or control groups on a 1:1 basis. The intervention group received behavioral and emotional support, whereas the control group received usual care. Interventions included motivational interviewing, patient activation, empathetic communication, addressing mental health and substance use, and mindfulness. PRINCIPAL FINDINGS: Observed total readmission costs were significantly lower in the intervention group than in the control group ($1.1 million vs. $2.0 million) as was the observed mean cost per readmitted patient ($44,052 vs. $91,278). The mean expected cost of readmission after adjustment for confounding variables was lower in the intervention group than in the control group ($8,094 vs. $9,882, p = .011). PRACTICAL APPLICATIONS: Readmissions are a costly spend category. In this study, posthospital discharge support addressing the psychosocial factors contributing to patients' readmissions resulted in a lower total cost of care for those with a cardiovascular diagnosis. We describe an intervention that is reproducible and can be scaled broadly through technology to reduce readmission costs.

The APPD Longitudinal Educational Assessment Research Network's First Decade.

ABSTRACT: In 2009, the Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN), a national educational research network, was formed. We report on evaluation of the network after 10 years of operation by reviewing program context, input, processes, and products to measure its progress in performing educational research that advances training of future pediatricians. Historical changes in medical education shaped the initial development of the network. APPD LEARN now includes 74% (148 of 201) of US Pediatric residency programs and has recently incorporated a network of Pediatric subspecialty fellowship programs. At the time of this evaluation, APPD LEARN had approved 19 member-initiated studies and 14 interorganizational studies, resulting in 23 peer-reviewed publications, numerous presentations, and 7 archived sharable data sets. Most publications focused on how and when interventions work rather than whether they work, had high scores for reporting rigor, and included organizational and objective performance outcomes. Member program representatives had positive perceptions of APPD LEARN's success, with most highly valuing participation in research that impacts training, access to expertise, and the ability to make authorship contributions for presentations and publication. Areas for development and improvement identified in the evaluation include adopting a formal research prioritization process, infrastructure changes to support educational research that includes patient data, and expanding educational outreach within and outside the network. APPD LEARN and similar networks contribute to high-rigor research in pediatric education that can lead to improvements in training and thereby the health care of children.

Pediatric Emergency Medicine Fellowship Program Directors' Viewpoint: Minimum Levels of Entrustment for Graduating Fellows and Practicing Physicians to Perform the Subspecialty's Professional Activities.

OBJECTIVES: Entrustable Professional Activities (EPAs) are essential tasks physicians perform within their professions. Entrustment levels that pediatric emergency medicine (PEM) fellowship program directors (FPDs) expect graduating fellows to achieve for PEM-specific and common pediatric subspecialty EPAs remain unreported. This study aims to determine minimum entrustment levels FPDs require fellows to achieve to graduate from fellowship and to compare FPD expectations for fellows versus practicing PEM physicians. METHODS: Secondary analysis of PEM-specific data from a national multispecialty cross-sectional survey of pediatric subspecialty FPDs. For 6 PEM-specific and 7 common pediatric subspecialty EPAs, PEM FPDs indicated (1) minimum entrustment levels fellows should achieve by training completion, (2) whether they would allow a fellow to graduate below these minimum levels, and (3) minimum levels for safe and effective practice by PEM physicians. Minimum levels were defined as the level that more than 80% of FPDs would not drop below. RESULTS: Sixty of 77 PEM FPDs (78%) completed the survey. Most respondents did not require fellows to achieve the highest level (level 5-no supervision) by graduation for any PEM-specific EPAs. The median level FPDs expected for practicing PEM physicians was 5 (trusted to perform without supervision) for EPAs 1 and 4 and level 4 (indirect supervision for complex cases) for the remaining PEM-specific EPAs. Minimum levels expected by FPDs for common subspecialty pediatric EPAs were lower for both groups. CONCLUSIONS: Most PEM FPDs indicated that they would graduate fellows before their achievement of the highest entrustment level for all EPAs. Most also indicated that they do not expect practicing PEM physicians to perform all EPAs without supervision. These findings indicate need for stakeholders to evaluate current structure and outcomes of PEM fellowship programs and for institutions and organizations to ensure adequate support in time and resources for ongoing learning for practicing PEM physicians.

Assessment of pleural pressure during sleep in Marfan syndrome.

STUDY OBJECTIVES: Patients with Marfan syndrome (MFS) have a high risk for aortic aneurysms. They are also susceptible to sleep-disordered breathing that may expose them to highly negative intrathoracic pressures known to increase aortic transmural pressure, which may accelerate aortic dilatation. Our objective was to quantify overnight intrathoracic pressure changes during sleep in snoring patients with MFS and the therapeutic effect of continuous positive airway pressure (CPAP). METHODS: We used a questionnaire to identify self-reported snoring patients with MFS. In these patients, we monitored intrathoracic pressure using esophageal pressure (Pes) during overnight baseline and CPAP sleep studies. We defined a peak-inspiratory Pes (Pespeak-insp) < - 5 cm H2O as greater than normal and examined the distribution of Pespeak-insp during baseline and CPAP studies. RESULTS: In our sample of 23 snorers with MFS, we found that 70% of sleep breaths exhibited Pespeak-insp < -5 cm H2O, with apnea/hypopneass accounting for only 12%, suggesting prevalent stable flow-limited breathing and snoring. In a subset (n = 12) with Pes monitoring during a CPAP night, CPAP lowered the mean proportion of breaths with Pespeak-insp < -5 cm H2O from 83.7% ± 14.9% to 3.6% ± 3.0% (P < .001). In addition, contemporaneous aortic root diameter was associated with the mean Pespeak-insp during inspiratory flow-limited breathing and apneas/hypopneas (ß = -0.05, r = .675, P = .033). CONCLUSIONS: The sleep state in MFS revealed prolonged exposure to exaggerated negative inspiratory Pes, which was reversible with CPAP. Since negative intrathoracic pressure can contribute to thoracic aortic stress and aortic dilatation, snoring may be a reversible risk factor for progression of aortic pathology in MFS. CITATION: Sowho M, Jun J, Sgambati F, et al. Assessment of pleural pressure during sleep in Marfan syndrome. J Clin Sleep Med. 2022;18(6):1583-1592.

Impact of an unannounced standardized veteran program on access to community-based services for veterans experiencing homelessness.

BACKGROUND: The United States Department of Veterans Affairs established a program in which actors incognito portray veterans experiencing homelessness with pre-determined needs to identify barriers to access and services at community-based organizations. METHODS: From 2017 to 2019, actors who varied in gender, skin color and age portrayed one of three scripts at all VA Community-Based Resource and Referral Centers (CRRCs) serving veterans experiencing homelessness in 30 cities and completed an evaluative survey. They carried authentic VA identification and were registered in a VA patient database for each identity. CRRCs were provided with reports annually and asked to implement corrective plans. Data from the survey were analysed for change over time. RESULTS: Access to food, counselling, PTSD treatment, and hypertension/prediabetes care services increased significantly from 68-77% in year 2 to 83-97% in year 3 (each P < 0.05 adjusted for script present). A significant disparity in access for African American actors resolved following more uniform adherence to pre-existing policies. CONCLUSIONS: The 'unannounced standardized veteran' (USV) can identify previously unrecognized barriers to needed services and care. Audit and feedback programs based on direct covert observation with systematic data collection and rapid feedback may be an effective strategy for improving services to highly vulnerable populations.

Development of an Entrustment-Supervision Assessment Tool for Pharmacy Experiential Education Using Stakeholder Focus Groups.

Objective To devise a pharmacy-specific, expanded entrustable professional activities (EPA) entrustment-supervision scale that would frame preceptor ratings in a prospective and retrospective manner for use in experiential learning settings.Methods A series of focus group sessions were conducted to solicit expert opinion on how to develop and refine two entrustment-supervision scales. Purposive sampling was used to identify experts from different professional groups (physicians, pharmacy experiential administrators, and pharmacy practice faculty) who had extensive knowledge regarding EPAs and at least one publication related to EPAs. Panelists were invited to participate via email. Three focus sessions were conducted via videoconferencing between June and September 2019. The primary outcome was development of a pharmacy-specific EPA entrustment-supervision assessment tool. Secondary outcomes were individual entrustment-supervision statements across five levels of the entrustment-supervision scale.Results The focus group consisted of four pharmacy practice faculty, two experiential administrators, and one academic physician. Four concepts emerged from the focus group discussion: need for more granularity in entrustment-supervision scales; limitations due to differences in licensure requirements across the United States; present and ongoing use of expanded scales by schools and colleges of pharmacy; and uncertainty regarding how to rate entrustment-supervision when a student exhibits unprofessional behavior.Conclusion A pharmacy-specific, expanded EPA entrustment-supervision scale will be useful to support longitudinal assessment of learners in experiential settings where an EPA framework is utilized. Determining when to use a prospective versus retrospective perspective requires further evaluation.

Telemedicine in Pediatric Training: A National Needs Assessment of the Current State of Telemedicine Education in Pediatric Training.

PURPOSE: To describe the current state of telemedicine within pediatric training programs to inform development of a national telemedicine training curriculum for pediatric trainees. METHODS: We conducted an anonymous cross-sectional survey of pediatric residency (Fall 2020) and fellowship program directors (Spring 2021) on their current telemedicine practices in pediatric post-graduate training. RESULTS: Forty-eight US pediatric residency programs (n = 48/198, 24%) and 422 fellowship programs completed the survey (n = 422/872, 48%); combined response rate 44% (n = 470/1070). Pre-COVID-19, 12% (n = 57/470) of programs surveyed reported using telemedicine in their training program, but during the pandemic 71% (n = 334/470) reported telemedicine use with trainees. Over 71% (n = 334/470) agreed that a formalized curriculum is important, yet 69% (n = 262/380) of programs reporting telemedicine use either did not have a curriculum or were unsure if one existed at their program. Respondents who were unsure/not likely to add a telemedicine curriculum and/or indicated that a telemedicine curriculum would not be important (52% n = 243/470), cited "time" (55%, n = 136/243) most frequently as a barrier. CONCLUSIONS: Our needs assessment indicates marked increase in use of telemedicine with trainees by respondent pediatric training programs, with fewer than 50% reporting a formalized training curriculum and most agreeing that a curriculum is important.

Fellow Entrustment for the Common Pediatric Subspecialty Entrustable Professional Activities Across Subspecialties.

OBJECTIVE: To determine the relationship between level of supervision (LOS) ratings for the Common Pediatric Subspecialty Entrustable Professional Activities (EPAs) with their associated subcompetency milestones across subspecialties and by fellowship training year. METHODS: Clinical Competency Committees (CCCs) in 14 pediatric subspecialties submitted LOS ratings for 6 Common Subspecialty EPAs and subcompetency milestone levels mapped to these EPAs. We examined associations between these subcompetency milestone levels and LOS ratings across subspecialty training year by fitting per-EPA linear mixed effects models, regressing LOS rating on milestone level and on training year. RESULTS: CCCs from 211 pediatric fellowship programs provided data for 369 first, 336 second, and 331 third year fellows. Mean subcompetency milestone levels increased similarly among subspecialties for most EPAs compared with the reference, Adolescent Medicine. Mean subcompetency milestones mapped to each EPA and mean EPA LOS ratings generally increased by training year across all subspecialties. CONCLUSIONS: Subcompetency milestones levels mapped to each Common Subspecialty EPA and the EPA LOS ratings increase similarly across subspecialties and by training year, providing validity evidence for using EPA LOS to assess pediatric subspecialty trainee performance. This study supports the development of tools to facilitated the CCC evaluation process across all pediatric subspecialties.

Continued Supervision for the Common Pediatric Subspecialty Entrustable Professional Activities May Be Needed Following Fellowship Graduation.

PURPOSE: Entrustable professional activities (EPAs) are one approach to competency-based medical education (CBME), and 7 EPAs have been developed that address content relevant for all pediatric subspecialties. However, it is not known what level of supervision fellowship program directors (FPDs) deem necessary for graduation. The Subspecialty Pediatrics Investigator Network (SPIN) investigated FPD perceptions of the minimum level of supervision required for a trainee to successfully graduate. METHOD: In 2017, SPIN surveyed all FPDs of accredited fellowships for 14 subspecialties. For each EPA, the minimum supervision level for graduation (ranging from observation only to unsupervised practice) was set such that no more than 20% of FPDs would accept a lower level. RESULTS: The survey response rate was 82% (660/802). The minimum supervision level for graduation varied across the 7 EPAs from 2 (direct) to 4 (indirect for complex cases), with significant differences between EPAs. The percentage of FPDs desiring a lower minimum supervision level ranged from 3% to 17%. Compared with the 4 nonclinical EPAs (quality improvement, management, lead within the profession, scholarship), the 3 clinical EPAs (consultation, handover, lead a team) had higher minimum supervision graduation levels (P < .001), with less likelihood that an FPD would graduate a learner below their minimum level (P < .001). CONCLUSIONS: Consensus among FPDs across all pediatric subspecialties demonstrates the potential need for ongoing supervision for graduates in all 7 common pediatric subspecialty EPAs after fellowship. As CBME programs are implemented, processes and infrastructure to support new graduates are important considerations for leaders.

Learner Levels of Supervision Across the Continuum of Pediatrics Training.

PURPOSE: To describe trajectories in level of supervision ratings for linked entrustable professional activities (EPAs) among pediatric learners in medical school, residency, fellowship. METHOD: The authors performed secondary analyses of 3 linked datasets of level of supervision ratings for the Core EPAs for Entering Residency, the General Pediatrics EPAs, and the Subspecialty Pediatrics EPAs. After identifying 9 activities in common across training stages and aligning the level of entrustment-supervision scales across the datasets, piecewise ordinal and linear mixed effects models were fitted to characterize trajectories of supervision ratings. RESULTS: Within each training period, learners were rated as needing less supervision over time in each activity. When transitioning from medical school to residency or during the first year of residency, learners were rated as needing greater supervision in activities related to patient management, teamwork, emergent care, and public health/QI than in earlier periods. When transitioning from residency to fellowship, learners were always rated as needing greater supervision than they had been accorded at the end of residency and sometimes even more than they had been accorded at the start of residency. CONCLUSIONS: Although development over training is often imagined as continuous and monotonically increasing competence, this study provides empirical evidence supporting the idea that entrustment is a set of discrete decisions. The relaxation of supervision in training is not a linear process. Even with a seamless curriculum, supervision is tightly bound to the training setting. Several explanations for these findings are discussed.

Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis.

BACKGROUND: Meaningful variations in physician performance are not always discernible from the medical record. OBJECTIVE: We used unannounced standardized patients to measure and provide feedback on care quality and fidelity of documentation, and examined downstream effects on reimbursement claims. DESIGN: Static group pre-post comparison study conducted between 2017 and 2019. SETTING: Fourteen New Jersey primary care practice groups (22 practices) enrolled in Horizon BCBS's value-based program received the intervention. For claims analyses, we identified 14 additional comparison practice groups matched on county, practice size, and claims activity. PARTICIPANTS: Fifty-nine of 64 providers volunteered to participate. INTERVENTION: Unannounced standardized patients (USPs) made 217 visits portraying patients with 1-2 focal conditions (diabetes, depression, back pain, smoking, or preventive cancer screening). After two baseline visits to a provider, we delivered feedback and conducted two follow-up visits. MEASUREMENTS: USP-completed checklists of guideline-based provider care behaviors, visit audio recordings, and provider notes were used to measure behaviors performed and documentation errors pre- and post-feedback. We also compared changes in 3-month office-based claims by actual patients between the intervention and comparison practice groups before and after feedback. RESULTS: Expected clinical behaviors increased from 46% to 56% (OR = 1.53, 95% CI 1.29-1.83, p < 0.0001), with significant improvements in smoking cessation, back pain, and depression screening. Providers were less likely to document unperformed tasks after (16%) than before feedback (18%; OR = 0.74, 95% CI 0.62 to 0.90, p = 0.002). Actual claim costs increased significantly less in the study than comparison group for diabetes and depression but significantly more for smoking cessation, cancer screening, and low back pain. LIMITATIONS: Self-selection of participating practices and lack of access to prescription claims. CONCLUSION: Direct observation of care identifies hidden deficits in practice and documentation, and with feedback can improve both, with concomitant effects on costs.