The Next Era of Assessment: Building a Trustworthy Assessment System.

Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.

Burnout During the COVID-19 Pandemic: A Report on Pediatric Residents.

BACKGROUND AND OBJECTIVE: The Pediatric Resident Burnout and Resilience Consortium (PRB-RSC) has described the epidemiology of burnout in pediatric residents since 2016. We hypothesized burnout rates during the pandemic would increase. We explored resident burnout during the COVID-19 pandemic and its relationship to resident perception of workload, training, personal life, and local COVID burden. METHODS: Since 2016, PRB-RSC has sent an annual, confidential survey to over 30 pediatric and medicine-pediatrics residencies. In 2020 and 2021, seven questions were added to explore the relationship of COVID-19 and perceptions of workload, training, and personal life. RESULTS: In 2019, 46 programs participated, 22 in 2020, and 45 in 2021. Response rates in 2020 (n = 1055, 68%) and 2021(n = 1702, 55%) were similar to those of previous years (P = .09). Burnout rates in 2020 were significantly lower than in 2019 (54% vs 66%, P < .001) but returned to pre-COVID levels in 2021 (65%, P = .90). In combined 2020-2021 data, higher rates of burnout were associated with reported increased workload (Adjusted Odds Ratio (AOR) 1.38, 95% CI 1.19-1.6) and concerns regarding the effect of COVID on training (AOR 1.35, 95% CI 1.2-1.53). Program-level county COVID burden in combined 2020-2021 data was not associated with burnout in this model (AOR=1.03, 95% CI 0.70-1.52). CONCLUSIONS: Burnout rates within reporting programs decreased significantly in 2020 and returned to prepandemic levels in 2021. Increased burnout was associated with perceived increases in workload and concerns regarding effect of the pandemic on training. Given these findings, programs should consider further investigation into workload and training uncertainty on burnout.

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine.

PURPOSE: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). METHOD: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified. RESULTS: 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations ( r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. CONCLUSIONS: For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.

Utilization of heat-mapping tools to match a resident staffing template to emergency department arrival patterns.

OBJECTIVES: Academic emergency departments (ED) rely on a steady flow of patients to provide residents with good clinical training. Understanding institutional volume patterns allows training directors to create a schedule that maximizes learning opportunities while also adequately staffing the ED. Our primary objective of this study was to utilize heat-mapping software to optimize resident staffing in an academic ED. METHODS: Heat-mapping tools within Microsoft Excel were utilized to overlay ED patient arrival patterns on top of the potential patients per hour based on published productivity data for trainees and historical averages for advanced practice providers at our institution. Time frames for under- and overstaffing were identified and color-coded. This analysis informed a revised schedule template and the same heat-mapping process was used to determine the appropriateness of the revised staffing template. RESULTS: The heat map for the original schedule template revealed understaffing in the morning and overstaffing the rest of the day. Informed by these findings, schedule adjustments were made. There was no net increase in the number of resident or advanced practice provider coverage hours. Prior to implementation, the ED was understaffed by 5% or more during 18.4% of operating hours. Following changes to the staffing template, only 5.9% of operating hours were understaffed (p < 0.001). Furthermore, significant understaffing (20% or more) decreased from 16.6% to 3.1% (p < 0.001). CONCLUSIONS: Novel use of heat-mapping software has the potential to successfully match ED patient arrival patterns to an optimal resident staffing template. Future directions include incorporation of variable resident productivity to account for fatigue as the shift progresses.

Invasive Bacterial Infections in Afebrile Infants Diagnosed With Acute Otitis Media.

OBJECTIVES: To determine the prevalence of invasive bacterial infections (IBIs) and adverse events in afebrile infants with acute otitis media (AOM). METHODS: We conducted a 33-site cross-sectional study of afebrile infants ≤90 days of age with AOM seen in emergency departments from 2007 to 2017. Eligible infants were identified using emergency department diagnosis codes and confirmed by chart review. IBIs (bacteremia and meningitis) were determined by the growth of pathogenic bacteria in blood or cerebrospinal fluid (CSF) culture. Adverse events were defined as substantial complications resulting from or potentially associated with AOM. We used generalized linear mixed-effects models to identify factors associated with IBI diagnostic testing, controlling for site-level clustering effect. RESULTS: Of 5270 infants screened, 1637 met study criteria. None of the 278 (0%; 95% confidence interval [CI]: 0%-1.4%) infants with blood cultures had bacteremia; 0 of 102 (0%; 95% CI: 0%-3.6%) with CSF cultures had bacterial meningitis; 2 of 645 (0.3%; 95% CI: 0.1%-1.1%) infants with 30-day follow-up had adverse events, including lymphadenitis (1) and culture-negative sepsis (1). Diagnostic testing for IBI varied across sites and by age; overall, 278 (17.0%) had blood cultures, and 102 (6.2%) had CSF cultures obtained. Compared with infants 0 to 28 days old, older infants were less likely to have blood cultures (P < .001) or CSF cultures (P < .001) obtained. CONCLUSION: Afebrile infants with clinician-diagnosed AOM have a low prevalence of IBIs and adverse events; therefore, outpatient management without diagnostic testing may be reasonable.

Predicting Hemolytic Uremic Syndrome and Renal Replacement Therapy in Shiga Toxin-producing Escherichia coli-infected Children.

BACKGROUND: Shiga toxin-producing Escherichia coli (STEC) infections are leading causes of pediatric acute renal failure. Identifying hemolytic uremic syndrome (HUS) risk factors is needed to guide care. METHODS: We conducted a multicenter, historical cohort study to identify features associated with development of HUS (primary outcome) and need for renal replacement therapy (RRT) (secondary outcome) in STEC-infected children without HUS at initial presentation. Children aged <18 years who submitted STEC-positive specimens between January 2011 and December 2015 at a participating study institution were eligible. RESULTS: Of 927 STEC-infected children, 41 (4.4%) had HUS at presentation; of the remaining 886, 126 (14.2%) developed HUS. Predictors (all shown as odds ratio [OR] with 95% confidence interval [CI]) of HUS included younger age (0.77 [.69-.85] per year), leukocyte count ≥13.0 × 103/µL (2.54 [1.42-4.54]), higher hematocrit (1.83 [1.21-2.77] per 5% increase) and serum creatinine (10.82 [1.49-78.69] per 1 mg/dL increase), platelet count <250 × 103/µL (1.92 [1.02-3.60]), lower serum sodium (1.12 [1.02-1.23 per 1 mmol/L decrease), and intravenous fluid administration initiated ≥4 days following diarrhea onset (2.50 [1.14-5.46]). A longer interval from diarrhea onset to index visit was associated with reduced HUS risk (OR, 0.70 [95% CI, .54-.90]). RRT predictors (all shown as OR [95% CI]) included female sex (2.27 [1.14-4.50]), younger age (0.83 [.74-.92] per year), lower serum sodium (1.15 [1.04-1.27] per mmol/L decrease), higher leukocyte count ≥13.0 × 103/µL (2.35 [1.17-4.72]) and creatinine (7.75 [1.20-50.16] per 1 mg/dL increase) concentrations, and initial intravenous fluid administration ≥4 days following diarrhea onset (2.71 [1.18-6.21]). CONCLUSIONS: The complex nature of STEC infection renders predicting its course a challenge. Risk factors we identified highlight the importance of avoiding dehydration and performing close clinical and laboratory monitoring.

Lactic Acidosis in a Critically Ill Patient: Not Always Sepsis.

Lactic acidosis in the emergency department and other hospital settings is typically due to tissue hypoxia with sepsis being the most common cause. However, in patients with persistently elevated lactate without evidence of inadequate oxygen delivery, type B lactic acidosis should be considered. We report the case of a 12-year-old boy with relapsed and refractory pre-B-cell acute lymphoblastic leukemia who presented in distress with tachycardia, history of fever, and diffuse abdominal tenderness. The patient had severe metabolic acidosis with elevated lactate upon arrival to the emergency department. Despite aggressive fluid resuscitation and intravenous antibiotics, the patient's acidosis worsened. Serial blood cultures were negative, and he was eventually diagnosed with type B lactic acidosis secondary to relapsed acute lymphoblastic leukemia.

Pediatric Adnexal Torsion: Not Just a Postmenarchal Problem.

BACKGROUND: Pediatric adnexal torsion is rare, can be challenging to recognize, and may result in ovarian loss. OBJECTIVE: We sought to identify and compare the defining characteristics of adnexal torsion in premenarchal and postmenarchal girls. METHODS: A retrospective cohort study was performed at a tertiary care children's hospital, including patients diagnosed postnatally with adnexal (ovarian or tubal) torsion between 1997 and 2013. Proportions were compared using relative risk regression. RESULTS: Adnexal torsion was found in 59 premenarchal and 43 postmenarchal girls. Abdominal pain was the most common chief complaint (54%). History included reports of pain (96%), vomiting (67%), and fever (19%). Excluding 12 patients with isolated tubal torsion and 19 with a teratoma, there were no statistically significant differences in ovarian loss in premenarchal vs. postmenarchal girls (47% and 25% respectively; relative risk [RR] = 1.8 [95% confidence interval {CI} 0.9-3.8]), left- vs. right-sided torsion (47% and 32%; RR = 1.5 [95% CI 0.8-2.7]), pain duration ≤ 2 days vs. > 2 days (31% and 41%; RR = 0.8 [95% CI 0.4-1.5]; n = 64) and severe pain vs. mild to moderate (38% and 33%; RR = 1.1 [95% CI 0.7-1.5]; n = 56). CONCLUSIONS: The diagnosis of pediatric adnexal torsion is difficult and often delayed. Pain and tenderness may not be isolated to a unilateral lower quadrant. Although traditionally considered a postmenarchal problem, in a pediatric academic emergency department adnexal torsion occurred with similar frequency in premenarchal and postmenarchal girls. The potential for organ salvage means that adnexal torsion should be considered in all females presenting with acute abdominal pain regardless of age or menstrual history.

Management of Toddler's Fractures in the Pediatric Emergency Department.

OBJECTIVES: To evaluate current practice in treatment of toddler's fractures, as well as subsequent healthcare utilization and complications. METHODS: Retrospective cohort study of children age 9 months to 3 years with a radiographically evident toddler's fracture diagnosed at a single academic pediatric emergency department (PED) from January 2008 to December 2012. Data collected included initial form of immobilization (if any), referral to orthopedic clinic, number of repeat radiographs obtained, presence of skin breakdown related to splinting or casting, and presence of other complications. RESULTS: Seventy-five patients were treated. Most patients were placed in splints or casts in the PED (66.7%) as opposed to controlled ankle motion (CAM) boot (24%) or no immobilization (9.3%). Splinted patients had a longer total duration of immobilization, higher rate of follow-up in orthopedic clinic, and greater number of repeat radiographs obtained than those in the CAM boot or no immobilization groups. Thirteen patients (17.3%) developed skin breakdown during their course of therapy; all of these patients had been placed in a splint or cast in the PED. No difference in PED return rates was observed between groups. CONCLUSIONS: There is wide variation in management of toddler's fractures within this single tertiary care PED. Given that these fractures are unlikely to displace and that complications of splinting and casting are not insignificant, this study suggests that immobilization may not be necessary for acute management of toddler's fractures.

Clinical practice guidelines for pediatric appendicitis evaluation can decrease computed tomography utilization while maintaining diagnostic accuracy.

OBJECTIVES: The objective of this study was to compare usage of computed tomography (CT) scan for evaluation of appendicitis in a children's hospital emergency department before and after implementation of a clinical practice guideline focused on early surgical consultation before obtaining advanced imaging. METHODS: A multidisciplinary team met to create a pathway to formalize the evaluation of pediatric patients with abdominal pain. Computed tomography scan utilization rates were studied before and after pathway implementation. RESULTS: Among patients who had appendectomy in the year before implementation (n = 70), 90% had CT scans, 6.9% had ultrasound, and 5.7% had no imaging. The negative appendectomy rate before implementation was 5.7%. In patients undergoing appendectomy in the postimplementation cohort (n = 96), 48% underwent CT, 39.6% underwent ultrasound, and 15.6% had no imaging. The negative appendectomy rate was 5.2%. We demonstrated a 41% decrease in CT use for patients undergoing appendectomy at our institution without an increase in the negative appendectomy rate or missed appendectomy. The results were even more striking when comparing the rate of CT scan use in the subset of patients undergoing appendectomy without imaging from an outside hospital. In these patients, CT scan utilization decreased from 82% to 20%, a 76% reduction in CT use in our facility after protocol implementation. CONCLUSIONS: Implementation of a clinical evaluation pathway emphasizing examination, early surgeon involvement, and utilization of ultrasound as the initial imaging modality for evaluation of abdominal pain concerning for appendicitis resulted in a marked decrease in the reliance on CT scanning without loss of diagnostic accuracy.