Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment.

BACKGROUND: The Trauma NOn-TECHnical Skills (T-NOTECHS) tool has been used to assess teamwork in trauma resuscitation, but its reliability and validity for self-assessment is unknown. Our purpose was to determine the reliability and validity of self-administered T-NOTECHS in pediatric trauma resuscitation. METHODS: Simulated in situ resuscitations were evaluated using T-NOTECHS in real time by experts and immediately afterwards by team members. Reliability was analyzed with linear-weighted kappa and intra-class correlation. T-NOTECHS scores were compared between expert (gold-standard) and self-assessment. RESULTS: Fifteen simulations were examined. T-NOTECHS scores were similar between self- and expert assessment for leadership. Self-assessment scores were higher than expert for the other domains and total composite score. Inter-rater reliability for total score was similar between the two groups, but differences were observed in the domains. CONCLUSIONS: Self-assessment is not interchangeable with expert rating when using T-NOTECHS. Future studies need to determine how self-assessment can be best utilized. LEVEL OF EVIDENCE: Studies of diagnostic accuracy - Level 2.

Qualitative assessment of simulation-based training for pediatric trauma resuscitation.

BACKGROUND: Effective teamwork is critical in the trauma bay, although there is a lack of consensus related to optimal training for these skills. We implemented in situ trauma simulations with debriefing as a possible training methodology to improve team-oriented skills. METHODS: Focus groups were conducted with multidisciplinary clinicians who respond to trauma activations. The focus group questions were intended to elicit discussion on the clinicians' experiences during trauma activations and simulations with an emphasis on confidence, leadership, cooperation, communication, and opportunities for improvement. Thematic content analysis was conducted using Atlas.ti analytical software. RESULTS: Ten focus groups were held with a total of 55 clinicians. Qualitative analysis of focus group feedback revealed the following selected themes: characteristics of a strong leader during a trauma, factors impacting trauma team members' confidence, and effective communication as a key component during trauma response. Participants recommended continued simulations to enhance trauma team trust and efficiency. CONCLUSION: Clinicians responding to pediatric trauma resuscitations valued the practice they received during trauma simulations and supported the continuation of the simulations to improve trauma activation teamwork and communication. Findings will inform the development of future simulation-based training programs to improve teamwork, confidence, and communication between trauma team members.

The effect of multimedia replacing text in resident clinical decision-making assessment.

Multimedia in assessing clinical decision-making skills (CDMS) has been poorly studied, particularly in comparison to traditional text-based assessments. The literature suggests multimedia is more difficult for trainees. We hypothesize that pediatric residents score lower in diagnostic skill when clinical vignettes use multimedia rather than text for patient findings. A standardized method was developed to write text-based questions from 60 high-resolution, quality multimedia; a series of expert panels selected 40 questions with both a multimedia and text-based counterpart, and two online tests were developed. Each test featured 40 identical questions with reciprocal and alternating modality (multimedia vs. text). Pediatric residents and rising 4th year medical students (MS-IV) at a single residency were randomized to complete either test stratified by postgraduate training year (PGY). A mixed between-within subjects ANOVA analyzed differences in score due to modality and PGY. Secondary analyses ascertained modality effect in dermatology and respiratory questions using Mann-Whitney U tests, and correlations on test performance to In-service Training Exam (ITE) scores using Spearman rank. Eighty-eight residents and rising interns completed the study. Overall multimedia scores were lower than text-based scores (p = 0.047, η p2  = 0.04), with highest disparity in rising interns (MS-IV); however, PGY had a greater effect on scores (p = 0.001, η p2  = 0.16). Respiratory questions were not significantly lower with multimedia (n = 9, median 0.71 vs. 0.86, p = 0.09) nor dermatology questions (n = 13, p = 0.41). ITEs correlated significantly with text-based scores (ρ = 0.23-0.25, p = 0.04-0.06) but not with multimedia scores. In physician trainees with less clinical experience, multimedia-based case vignettes are associated with significantly lower scores. These results help shed light on the role of multimedia versus text-based information in CDMS, particularly in less experienced clinicians.