Validation of a performance checklist for ultrasound-guided internal jugular central lines for use in procedural instruction and assessment.

PURPOSE OF THE STUDY: Tools created to measure procedural competency must be tested in their intended environment against an established standard in order to be validated. We previously created a checklist for ultrasound-guided internal jugular central venous catheter (US IJ CVC) insertion using the modified Delphi method. We sought to further validate the checklist tool for use in an educational environment. STUDY DESIGN: This is a cohort study involving 15 emergency medicine interns being evaluated on their skill in US IJ CVC placement. We compared the checklist tool with a modified version of a clinically validated global rating scale (GRS) for procedural performance. RESULTS: The correlation between the GRS tool and the checklist tool was excellent, with a correlation coefficient (Pearson's r) of 0.90 (p<0.0001). CONCLUSIONS: This checklist represents a useful tool for measuring procedural competency.

Derivation of a Performance Checklist for Ultrasound-Guided Arthrocentesis Using the Modified Delphi Method.

OBJECTIVES: Arthrocentesis is an important skill for physicians in multiple specialties. Recent studies indicate a superior safety and performance profile for this procedure using ultrasound guidance for needle placement, and improving quality of care requires a valid measurement of competency using this modality. METHODS: We endeavored to create a validated tool to assess the performance of this procedure using the modified Delphi technique and experts in multiple disciplines across the United States. RESULTS: We derived a 22-item checklist designed to assess competency for the completion of ultrasound-guided arthrocentesis, which demonstrated a Cronbach's alpha of 0.89, indicating an excellent degree of internal consistency. CONCLUSIONS: Although we were able to demonstrate content validity for this tool, further validity evidence should be acquired after the tool is used and studied in clinical and simulated contexts.

Video-augmented feedback for procedural performance.

BACKGROUND: Resident programs must assess residents' achievement of core competencies for clinical and procedural skills. AIMS: Video-augmented feedback may facilitate procedural skill acquisition and promote more accurate self-assessment. METHODS: A randomized controlled study to investigate whether video-augmented verbal feedback leads to increased procedural skill and improved accuracy of self-assessment compared to verbal only feedback. Participants were evaluated during procedural training for ultrasound guided internal jugular central venous catheter (US IJ CVC) placement. All participants received feedback based on a validated 30-point checklist for US IJ CVC placement and validated 6-point procedural global rating scale. RESULTS: Scores in both groups improved by a mean of 9.6 points (95% CI: 7.8-11.4) on the 30-point checklist, with no difference between groups in mean score improvement on the global rating scale. In regards to self-assessment, participant self-rating diverged from faculty scoring, increasingly so after receiving feedback. Residents rated highly by faculty underestimated their skill, while those rated more poorly demonstrated increasing overestimation. CONCLUSIONS: Accuracy of self-assessment was not improved by addition of video. While feedback advanced the skill of the resident, video-augmented feedback did not enhance skill acquisition or improve accuracy of resident self-assessment compared to standard feedback.

Chronic Salicylate Toxicity Simulation.

Introduction: Chronic salicylate toxicity is an uncommon, potentially life-threatening poisoning that requires high clinical suspicion in order to make the diagnosis. We created a simulation case that challenges learners to analyze case information, construct a differential diagnosis of an elevated anion gap metabolic acidosis with respiratory alkalosis, and initiate treatment for this toxicity. Methods: The simulation case was designed for emergency medicine residents and pediatric emergency medicine fellows. The activity began with a brief overview of the monitors, equipment, and simulation experience. For interns, a team of two learners comanaged the case; for senior learners, the case was managed solo. The learners had 15 minutes to complete a focused history and physical exam, request and interpret labs and studies, and initiate specific treatments. The simulation was followed by a 15-minute facilitated debrief session that included an overview of key learning points and learner performance based on an evaluation checklist. Results: Residents completed a postparticipation questionnaire consisting of six questions rated on a 5-point Likert scale. Overall, residents reported a high degree of satisfaction with the simulation experience. The case and debrief were effective in meeting the educational objectives and proved to be an effective modality to fill this educational gap. Discussion: This simulation exercise was effective in showing residents the uncommon presentation of chronic salicylate toxicity. Learners reported increased confidence in recognizing and managing this ingestion. The simulation experience closed an identified education gap and provided an experiential learning opportunity that accomplished the targeted learning objectives.

Toxic Alcohol Ingestion/Methanol Ingestion.

Introduction: Methanol poisoning is an uncommon life-threatening ingestion associated with significant morbidity and requires prompt diagnosis and management for the best possible outcome. We created a simulation case that challenges learners to analyze case information, construct a differential diagnosis of an anion gap metabolic acidosis, narrow the differential based on reasoning, and empirically initiate management. Methods: The simulation case was designed for emergency medicine residents and pediatric emergency medicine fellows. The activity began with a brief overview of the monitors, equipment, and simulation experience. First-year residents managed the case as a team of two. Second- and third-year residents and fellows managed the case alone. The learners had 15 minutes to complete a focused history and physical exam, request and interpret labs and studies, provide stabilization of life threats, and initiate specific interventions based on a presumptive diagnosis. The simulation was followed by a 20-minute facilitated debrief session that reviewed key learning points and learner performance based on an evaluation checklist. Results: Residents completed a six-question, 5-point Likert-scale postparticipation questionnaire. Overall, residents reported a high degree of satisfaction with the simulation experience. The case and debrief were effective in meeting the educational objectives and proved to be an effective modality to fill this educational gap. Discussion: This simulation experience successfully exposed residents to the uncommon presentation of methanol poisoning. The simulation experience effectively closed the identified educational gap and provided an experiential learning opportunity that accomplished the targeted learning objectives.

Delphi Method Validation of a Procedural Performance Checklist for Insertion of an Ultrasound-Guided Internal Jugular Central Line.

Placement of ultrasound-guided central lines is a critical skill for physicians in several specialties. Improving the quality of care delivered surrounding this procedure demands rigorous measurement of competency, and validated tools to assess performance are essential. Using the iterative, modified Delphi technique and experts in multiple disciplines across the United States, the study team created a 30-item checklist designed to assess competency in the placement of ultrasound-guided internal jugular central lines. Cronbach α was .94, indicating an excellent degree of internal consistency. Further validation of this checklist will require its implementation in simulated and clinical environments.

A controlled evaluation of case clinical effect coding by poison center specialists for detection of WMD scenarios.

BACKGROUND: Many public health entities employ computer-based syndromic surveillance to monitor for aberrations including possible exposures to weapons of mass destruction (WMD). Often, this is done by screening signs and symptoms reported for cases against syndromic definitions. Poison centers (PCs) may offer significant contributions to public health surveillance because of their detailed clinical effect data field coding and real-time data entry. Because improper clinical effect coding may impede syndromic surveillance, it is important to assess this accuracy for PCs. METHODS: An AAPCC-certified regional PC assessed the accuracy of clinical effect coding by specialists in poison information (SPIs) listening to audio recordings of standard cases. Eighteen different standardized cases were used, consisting of six cyanide, six botulism, and six control cases. Cases were scripted to simulate clinically relevant telephone conversations and converted to audio recordings. Ten SPIs were randomly selected from the center's staff to listen to and code case information from the recorded cases. Kappa scores and the percentage of correctly coding a present clinical effect were calculated for individual clinical effects summed over all test cases along with corresponding 95% confidence intervals. The rate of the case coding by the SPIs triggering the PC's automated botulism and cyanide alerts was also determined. RESULTS: The kappa scores and the percentage of correctly coding a present clinical effect varied depending on the specific clinical effect, with greater accuracy observed for the clinical effects of vomiting and agitation/irritability, and poor accuracy observed for the clinical effects of visual defect and anion gap increase. Lack of correct coding resulted in only 60 and 86% of the cases that met the botulism and cyanide surveillance definitions, respectively, triggering the corresponding alert. There was no difference observed in the percentage of coding a present clinical effect between certified (9.0 years experience) and non-certified (2.4 years experience) specialists. There were no cases of coding errors that resulted in the triggering of a false positive alert. CONCLUSION: The success of syndromic surveillance depends on accurate coding of signs and symptoms. Although PCs generally contribute high-quality data to public health surveillance, it is important to recognize this potential weak link in surveillance methods.