Validation of Ultrasound Guidance Targeting Tasks: Relationship of Performance and Clinical Experience.

OBJECTIVE: To assess whether health care provider performance on a novel task trainer and a set of newly created ultrasound guidance targeting tasks relates with their level of clinical experience. METHODS: In a prospective cohort study, two assessment sessions were carried out in which inexperienced (n=21 and n=15) and experienced (n=14 and n=10) health care providers completed five previously described ultrasound guidance targeting tasks. Raw completion time, number of targeting errors, and error-adjusted completion time for each task were compared between groups with a t test for independent measures. Receiver operating characteristic analyses were performed to estimate whether error-adjusted completion time or number of errors could accurately differentiate between groups. RESULTS: Raw completion time was similar between groups. The number of errors and error-adjusted completion time were significantly lower in the experienced health care providers in all but the out-of-plane dowel task. The receiver operating characteristic analyses of number of errors demonstrated very high areas under the curve (0.93-0.98), sensitivities (100%), specificities (76-86%), positive likelihood ratios (4.2-7.0), and negative likelihood ratios (<0.001) indicating excellent differentiation between experienced and inexperienced health care providers. These values were notably lower in the receiver operating characteristic analyses of error-adjusted completion times. CONCLUSION: Task performance related well with a health care provider's level of clinical experience providing evidence of construct validity. For each task, we were able to determine a cutoff for number of errors that categorized experienced and inexperienced health care providers with very favorable sensitivity, specificity, positive likelihood ratios, and negative likelihood ratios. Our findings suggest it may be possible to use these cutoffs to objectively classify trainees as competent or not competent.

Development and Assessment of a Novel Task Trainer and Targeting Tasks for Ultrasound-guided Invasive Procedures.

RATIONALE AND OBJECTIVES: The American Institute of Ultrasound in Medicine (AIUM) recommends that all providers performing ultrasound-guided invasive procedures be competent in a core set of guidance skills common to all ultrasound-guided procedures, including in-plane and out-of-plane needle guidance and needle imaging optimization techniques such as probe translation, rotation, and heel-toe standoff. To allow for the practice and assessment of these core skills, we have created a novel task trainer and set of targeting tasks, and sought to obtain validity evidence in the content and response process domains for this training and assessment system according to the Standards for Educational and Psychological Testing. MATERIALS AND METHODS: We have constructed an ultrasound-guided invasive procedure training system and five targeting tasks that focused on the needle guidance skills outlined by the AIUM. All tasks were performed by obstetrics and gynecology or maternal-fetal medicine physicians with and without experience in ultrasound-guided invasive procedures during a series of simulation workshops. All participants completed a survey regarding the trainer's and the tasks' usefulness in the training of inexperienced physicians. RESULTS: The physicians who completed the tasks had favorable views of task trainer and curriculum. The targeting curriculum was felt to allow practice of all of the core guidance skills outlined by the AIUM. The average response provided for all of the tasks was 4.0 or greater, with half of the items having an average response of 4.5 or higher. CONCLUSIONS: We have constructed a task trainer that incorporates all of the core skills outlined by the AIUM. All five tasks received very favorable reviews from both experienced and inexperienced providers. Taken together, our findings suggest they have strong content and response process validity evidence.

Effectiveness of Vaginal Delivery Simulation in Novice Trainees.

OBJECTIVE: Simulation training has been demonstrated to increase medical student confidence with vaginal deliveries; however, effect on skill performance is still lacking. To determine if integration of simulation training into the OB/GYN clerkship improves performance of vaginal deliveries, we assessed the effectiveness of simulation in third-year medical students. METHODS: During the OB/GYN clerkship, third-year students were assigned to receive vaginal delivery simulation (n=54) or cervical exam simulation (n=56), with each group serving as a simulation naïve control for the other skill. As a final assessment of their skill, students performed a simulated vaginal delivery scored by a blinded observer using a procedural checklist (score 0-30). A satisfactory score was considered 26 or greater. The individual scores and percentage of satisfactory scores were compared between both groups using a Mann-Whitney U test and chi-square test, respectively. RESULTS: Vaginal delivery students had a significantly higher mean score (27 +/- 3.2) and percentage of students achieving a passing score (85%) than the cervical exam students (22 +/- 3.5 and 15%). There were no differences in vaginal delivery performance based on gender, nor was there any difference in the number of real-life deliveries performed between vaginal delivery and cervical exam students. CONCLUSIONS: Even though medical students had an equivalent clinical rotation experience, a short period of simulation training had a marked effect on their end-of-rotation performance. During initial resident or midwife training more than 5 hours of simulation will likely be required to properly prepare 100% of trainees.

Effectiveness of Labor Cervical Examination Simulation in Medical Student Education.

OBJECTIVE: To estimate whether simulation training improves medical students' cervical examination accuracy. BACKGROUND: The training paradigm for the labor cervical examination exposes patients to additional examinations, lacks a gold standard, and does not objectively assess trainee competence. To address these issues and optimize training, we assessed the effectiveness of cervical examination simulation in third-year medical students. METHODS: During the obstetrics and gynecology clerkship, a cohort study was performed in which third-year students were assigned to receive cervical examination simulation (n=50) or vaginal delivery simulation (n=48), with each group serving as a simulation-naive control for the other skill. As a final assessment, students performed 10 cervical examinations using task trainers. Exact accuracy and accuracy within 1 cm were compared between groups. Cumulative summation analyses were performed on the cervical examination group to assess competence and the average number of repetitions needed to achieve it. RESULTS: Cervical examination students were significantly more accurate (Mann-Whitney, P<.001) in assessing dilation (73% exact, 98% within 1 cm) and effacement (83% and 100%) than vaginal delivery students (dilation 52% and 82%, effacement 51% and 96%). In the cumulative summation analyses, 65-100% of students attained competence during the clerkship depending on the level of accuracy and cervical parameter assessed. On average, competence was achieved with 27-44 repetitions. CONCLUSION: Simulation training dramatically improved student accuracy in labor cervical examinations. Because not all students achieved competence, the cumulative summation analyses suggest that more than 100 repetitions would be needed if the goal was for the entire class of students to achieve competence. LEVEL OF EVIDENCE: II.

Impact of an Inexpensive Anatomy-Based Fetal Pig Simulator on Obstetric Ultrasound Training.

OBJECTIVES: The purpose of this study was to construct an inexpensive anatomy-based obstetric ultrasound task trainer and investigate whether introduction of this trainer into a hands-on obstetric ultrasound course improved course participants' ultrasound scanning skills. METHODS: The trainer was created by placing fetal pigs into preservative-filled heat-sealed polyethylene bags. Twenty-four participants in an obstetric ultrasound course at Wake Forest School of Medicine were randomized to receive hands-on scanning with pregnant women or hands-on scanning and fetal pig simulation. Biometric scans were performed before and after the course. The time to complete the scans, margin of error of biometric measurements, and number of technically adequate images per scan were compared between groups. RESULTS: Twelve participants were randomized into each group. Although a direct comparison of postcourse biometric scans demonstrated no difference between groups, participants that received simulation training showed significant improvements in the time to complete the biometric scan (P < .05) and number of technically adequate images obtained (P < .05), whereas those who did not receive simulation training did not show significant improvements. CONCLUSIONS: Addition of the fetal pig ultrasound task trainer resulted in improvements in the course participants' scanning efficiency even after very limited exposure. Incorporating the task trainer earlier and more broadly into obstetric ultrasound training may benefit trainees.

Development of a novel task trainer for second trimester ultrasound-guided uterine evacuation.

INTRODUCTION: Second trimester surgical termination of pregnancy by dilation and evacuation (D&E) is a procedure that the Obstetrics and Gynecology (OB/GYN) Residency Review Committee requires training programs to offer to their residents and that many residents desire to learn. However, because of limited numbers of credentialed providers, clinical training opportunities may be limited or not available at their training institutions. Simulated procedures may allow for residents and other learners to be more prepared and capable at the time of their first clinical procedures or advanced skill acquisition. METHODS: We describe the construction of a low-cost task trainer for simulation of ultrasound-guided second trimester uterine evacuation procedures using a preserved pig heart and fetal pig. Residents of OB/GYN used the task trainer and completed a survey to determine its effectiveness as a teaching tool. RESULTS: With the use of a 5-point Likert scale survey (1 inferior, 5 superior), 13 OB/GYN trainees felt that the task trainer reliably reproduced fetal extraction (4.08), would improve their clinical skills (4.46), and would allow them to obtain the skills required to perform the task in real life (4.38). The similarity of ultrasound images to real-life images had a mean score of 3.69. Additional comments about the usefulness of this task trainer in learning the skills necessary for D&E procedures from the participants were overall positive. CONCLUSIONS: The task trainer for ultrasound-guided second trimester uterine evacuation showed excellent face validity. This low-cost task trainer provides residents and other trainees with a tool to practice ultrasound-guided fetal extraction skills and improve proficiency with D&E outside the operating room. In addition, it increases their confidence in being able to perform the procedure on patients.