Practice Doesn't Make Perfect: Clinical Experience With Procedures Does Not Correlate Well With Competence in Third-Year Medical Students.

OBJECTIVE: There is limited data available about factors which promote competence with procedures in medical students. Specifically, the relationship between procedural clinical experience and performance on an assessment is unclear. We sought to determine whether a correlation exists between the amount and type of clinical experience with a procedure and student performance on a standardized assessment of that procedure. DESIGN: Faculty performed standardized assessments of third-year medical students on ten procedures using simulation. We prospectively surveyed students about 3 types of experience (performed, observed, and simulated) with these procedures during their clerkships. We then analyzed whether a correlation exists between student experience and their competency assessment scores using Pearson's correlation. SETTING/PARTICIPANTS: Third-year medical students at the University of Kentucky College of Medicine. RESULTS: In 2018 to 2019, 131 students were assessed on procedural competency with 10 failures. One hundred and twenty students (91.6%) completed the clinical experience survey. Correlations between types of experience and competency scores were small to moderate, with only 5 of 40 being significant. We found no correlation between experience having performed a procedure and competency score. CONCLUSIONS: Overall, we did not find convincing evidence of a correlation between experience with procedures during clerkships and performance on a competency assessment. This suggests other factors may be contributing to procedural competence, which has implications for how educators should develop procedural competence in students.

Current Practices and Perspectives on Peer Observation and Feedback: A National Survey.

OBJECTIVE: Peer observation and feedback (POF) is the direct observation of an activity performed by a colleague followed by feedback with the goal of improved performance and professional development. Although well described in the education literature, the use of POF as a tool for development beyond teaching skills has not been explored. We aimed to characterize the practice of POF among pediatric hospitalists to explore the perceived benefits and barriers and to identify preferences regarding POF. METHODS: We developed a 14-item cross-sectional survey regarding divisional expectations, personal practice, perceived benefits and barriers, and preferences related to POF. We refined the survey based on expert feedback, cognitive interviews, and pilot testing, distributing the final survey to pediatric hospitalists at 12 institutions across the United States. RESULTS: Of 357 eligible participants, 198 (56%) responded, with 115 (58%) practicing in a freestanding children's hospital. Although 61% had participated in POF, less than one half (42%) reported divisional POF expectation. The most common perceived benefits of POF were identifying areas for improvement (94%) and learning about colleagues' teaching and clinical styles (94%). The greatest perceived barriers were time (51%) and discomfort with receiving feedback from peers (38%), although participation within a POF program reduced perceived barriers. Most (76%) desired formal POF programs focused on improving teaching skills (85%), clinical management (83%), and family-centered rounds (82%). CONCLUSIONS: Although the majority of faculty desired POF, developing a supportive environment and feasible program is challenging. This study provides considerations for improving and designing POF programs.

Facilitating the Timely Discharge of Well Newborns by Using Quality Improvement Methods.

BACKGROUND AND OBJECTIVES: Discharges are a key driver of hospital throughput. Our pediatric hospitalist team sought to improve newborn nursery throughput by increasing the percentage of newborns on our service with a discharge order by 11 am. We hypothesized that implementing a discharge checklist would result in earlier discharge times for newborns who met discharge criteria. METHODS: We identified barriers to timely discharge through focus groups with key stakeholders, chart reviews, and brainstorming sessions. We subsequently created and implemented a discharge checklist to identify and address barriers before daily rounds. We tracked mean monthly discharge order times. Finally, we performed chart reviews to determine causes for significantly delayed discharge orders and used this information to modify rounding practices during a second plan-do-study-act cycle. RESULTS: During the 2-year period before the intervention, 24% of 3224 newborns had a discharge order entered by 11 am. In the 20 months after the intervention, 39% of 2739 newborns had a discharge order by 11 am, a 63% increase compared with the baseline. Observation for group B Streptococcus exposure was the most frequent reason for a late discharge order. CONCLUSIONS: There are many factors that affect the timely discharge of well newborns. The development and implementation of a discharge checklist improved our ability to discharge newborns on our pediatric hospitalist service by 11 am. Future studies to identify nonphysician barriers to timely newborn discharges may lead to further improvements in throughput between the labor and delivery and maternity suites units.

Are Graduating Pediatric Residents Prepared to Perform Infant Lumbar Punctures?: A Multi-Institutional Descriptive Study.

OBJECTIVES: Few published studies describe graduating pediatric residents' procedural skills or success rates. This information would help guide supervisors' decisions about graduating residents' preparedness, training, and supervision needs. This study aimed to measure success rates for graduating pediatric residents performing infant lumbar puncture (LP) during the final months of their training and to describe their experiences performing and supervising infant LPs during the course of their training. METHODS: This survey-based study was conducted at 10 academic medical institutions in 2013. The survey consisted of 4 domains: (1) demographics, (2) exposure to infant LP training as an intern, (3) number of LPs performed and supervised during residency, and (4) specific information on the most recent clinical infant LP. RESULTS: One hundred ninety-eight (82%) of 242 eligible graduating residents responded to the survey. A 54% success rate was noted for graduating residents when they were the first provider performing the infant LPs. Success rates were 24% if they were not the first provider to attempt the LP. Overall, graduating residents were supervised on 29% of their LPs, used anesthesia for 29%, and used the early stylet removal technique for 63%. The graduating residents performed a median of 12 infant LPs and supervised others on a median of 5 infant LPs throughout their residency. The vast majority reported feeling confident and prepared to perform this procedure. CONCLUSIONS: At the end of residency, graduating pediatric residents were rarely supervised and had low infant LP success rates despite confidence in their skills. However, graduating residents frequently supervised others performing this procedure.

Script Concordance Testing to Determine Infant Lumbar Puncture Practice Variation.

OBJECTIVES: Script concordance testing (SCT) is used to assess clinical decision-making. We explore the use of SCT to (1) quantify practice variations in infant lumbar puncture (LP) and (2) analyze physician's characteristics affecting LP decision making. METHODS: Using standard SCT processes, a panel of pediatric subspecialty physicians constructed 15 infant LP case vignettes, each with 2 to 4 SCT questions (a total of 47). The vignettes were distributed to pediatric attending physicians and fellows at 10 hospitals within the INSPIRE Network. We determined both raw scores (tendency to perform LP) and SCT scores (agreement with the reference panel) as well as the variation with participant factors. RESULTS: Two hundred twenty-six respondents completed all 47 SCT questions. Pediatric emergency medicine physicians tended to select LP more frequently than did general pediatricians, with pediatric emergency medicine physicians showing significantly higher raw scores (20.2 ± 10.2) than general pediatricians (13 ± 15; 95% confidence interval for difference, 1, 13). Concordance with the reference panel varied among subspecialties and by the frequency with which practitioners perform LPs in their practices. CONCLUSION: Script concordance testing questions can be used as a tool to detect subspecialty practice variation. We are able to detect significant practice variation in the self-report of use of LP for infants among different pediatric subspecialties.

Script concordance testing: assessing residents' clinical decision-making skills for infant lumbar punctures.

PURPOSE: Residents must learn which infants require a lumbar puncture (LP), a clinical decision-making skill (CDMS) difficult to evaluate because of considerable practice variation. The authors created an assessment model of the CDMS to determine when an LP is indicated, taking practice variation into account. The objective was to detect whether script concordance testing (SCT) could measure CDMS competency among residents for performing infant LPs. METHOD: In 2011, using a modified Delphi technique, an expert panel of 14 attending physicians constructed 15 case vignettes (each with 2 to 4 SCT questions) that represented various infant LP scenarios. The authors distributed the vignettes to residents at 10 academic pediatric centers within the International Simulation in Pediatric Innovation, Research, and Education Network. They compared SCT scores among residents of different postgraduate years (PGYs), specialties, training in adult medicine, LP experience, and practice within an endemic Lyme disease area. RESULTS: Of 730 eligible residents, 102 completed 47 SCT questions. They could earn a maximum score of 47. Median SCT scores were significantly higher in PGY-3s compared with PGY-1s (difference: 3.0; 95% confidence interval [CI] 1.0-4.9; effect size d = 0.87). Scores also increased with increasing LP experience (difference: 3.3; 95% CI 1.1-5.5) and with adult medicine training (difference: 2.9; 95% CI 0.6-5.0). Residents in Lyme-endemic areas tended to perform more LPs than those in nonendemic areas. CONCLUSIONS: SCT questions may be useful as an assessment tool to determine CDMS competency among residents for performing infant LPs.

Primary care physicians and hospitalists: two branches of the same tree.

The success of the evolving filed of pediatric hospitalist medicine should be judged on the health outcomes achieved for the more than 6 million children who are hospitalized annually. The focused approach hospitalists bring to defining the best knowledge that their role is important but is limited in the overall health of most children. In order to achieve the best health outcomes, hospitalists must fully partner with the child's primary care provider. By consistently communicating well during pre-admission, hospitalization, and discharge intervals, hospitalists and primary care pediatricians can enhance learning, as well as maximize outcomes for shared patients.