Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions.

OBJECTIVE: Currently, there is a worldwide shift toward competency-based medical education. This necessitates the use of automated skills assessment methods during self-guided interventions training. Making assessment methods that are transparent and configurable will allow assessment to be interpreted into instructional feedback. The purpose of this work is to develop and validate skills assessment methods in ultrasound-guided interventions that are transparent and configurable. METHODS: We implemented a method based upon decision trees and a method based upon fuzzy inference systems for technical skills assessment. Subsequently, we validated these methods for their ability to predict scores of operators on a 25-point global rating scale in ultrasound-guided needle insertions and their ability to provide useful feedback for training. RESULTS: Decision tree and fuzzy rule-based assessment performed comparably to state-of-the-art assessment methods. They produced median errors (on a 25-point scale) of 1.7 and 1.8 for in-plane insertions and 1.5 and 3.0 for out-of-plane insertions, respectively. In addition, these methods provided feedback that was useful for trainee learning. Decision tree assessment produced feedback with median usefulness 7 out of 7; fuzzy rule-based assessment produced feedback with median usefulness 6 out of 7. CONCLUSION: Transparent and configurable assessment methods are comparable to the state of the art and, in addition, can provide useful feedback. This demonstrates their value in self-guided interventions training curricula.

Cognitive load theory as a framework for simulation-based, ultrasound-guided internal jugular catheterization training: Once is not enough.

OBJECTIVE: The main objective of this study was to use the principles of cognitive load theory to design a curriculum that incorporates a progressive part practice approach to teaching ultrasound-guided (USG) internal jugular catheterization (IJC) to novices. A secondary objective was to compare the technical proficiency of residents trained using this curriculum with the technical proficiency of residents trained with the current local standard of a single simulation session. METHODS: The experimental group included 16 residents who attended three 2-hour sessions of progressive part practice in a simulation lab. The control group included 46 residents who attended the current local standard of a single 2-hour simulation session just prior to their intensive care unit rotation. Technical proficiency was assessed using hand motion analysis and time to procedure completion. RESULTS: After three sessions, median scores for right hand motion (RHM) (34.5; [27.0-49.0]), left hand motion (LHM) (35.5; [20.0-45.0]), and total time (TT) (117.0 s; [82.7-140.0]) in the experimental group were significantly better than the control group (p<0.001). Results for eight experimental group residents who were assessed for retention at a later date revealed median scores for RHM (45.0; [32.0-58.0]), LHM (33.5; [20.0-63.0]), and TT (150.0 s; [103.0-399.6]), which were significantly better than those of the control group (p=0.01, p<0.01, and p=0.02, respectively). CONCLUSION: These results support multiple sessions of progressive part practice in a simulation lab as an effective competency-based approach to teaching USG IJC in preparation for the clinical setting.

Development and Evaluation of a Simulation-based Curriculum for Ultrasound-guided Central Venous Catheterization.

OBJECTIVE: To develop a simulation-based curriculum for residents to learn ultrasound-guided (USG) central venous catheter (CVC) insertion, and to study the volume and type of practice that leads to technical proficiency. METHODS: Ten post-graduate year two residents from the Departments of Emergency Medicine and Anesthesiology completed four training sessions of two hours each, at two week intervals, where they engaged in a structured program of deliberate practice of the fundamental skills of USG CVC insertion on a simulator. Progress during training was monitored using regular hand motion analysis (HMA) and performance benchmarks were determined by HMA of local experts. Blinded assessment of video recordings was done at the end of training to assess technical competence using a global rating scale. RESULTS: None of the residents met any of the expert benchmarks at baseline. Over the course of training, the HMA metrics of the residents revealed steady and significant improvement in technical proficiency. By the end of the fourth session six of 10 residents had faster procedure times than the mean expert benchmark, and nine of 10 residents had more efficient left and right hand motions than the mean expert benchmarks. Nine residents achieved mean GRS scores rating them competent to perform independently. CONCLUSION: We successfully developed a simulation-based curriculum for residents learning the skills of USG CVC insertion. Our results suggest that engaging residents in three to four distributed sessions of deliberate practice of the fundamental skills of USG CVC insertion leads to steady and marked improvement in technical proficiency with individuals approaching or exceeding expert level benchmarks.

Computerized training system for ultrasound-guided lumbar puncture on abnormal spine models: a randomized controlled trial.

PURPOSE: A randomized controlled trial was carried out to determine whether Perk Tutor, a computerized training platform that displays an ultrasound image and real-time needle position in a three-dimensional (3D) anatomical model, would benefit residents learning ultrasound-guided lumbar puncture (LP) in simulation phantoms with abnormal spinal anatomy. METHODS: Twenty-four residents were randomly assigned to either the Perk Tutor (P) or the Control (C) group and asked to perform an LP with ultrasound guidance on part-task trainers with spinal pathology. Group P was trained with the 3D display along with conventional ultrasound imaging, while Group C used conventional ultrasound only. Both groups were then tested solely with conventional ultrasound guidance on an abnormal spinal model not previously seen. We measured potential tissue damage, needle path in tissue, total procedure time, and needle insertion time. Procedural success rate was a secondary outcome. RESULTS: The needle tracking measurements (expressed as median [interquartile range; IQR]) in Group P vs Group C revealed less potential tissue damage (39.7 [21.3-42.7] cm(2) vs 128.3 [50.3-208.2] cm(2), respectively; difference 88.6; 95% confidence intervals [CI] 24.8 to 193.5; P = 0.01), a shorter needle path inside the tissue (426.0 [164.9-571.6] mm vs 629.7 [306.4-2,879.1] mm, respectively; difference 223.7; 95% CI 76.3 to 1,859.9; P = 0.02), and lower needle insertion time (30.3 [14.0-51.0] sec vs 59.1 [26.0-136.2] sec, respectively; difference 28.8; 95% CI 2.2 to 134.0; P = 0.05). Total procedure time and overall success rates between groups did not differ. CONCLUSION: Residents trained with augmented reality 3D visualization had better performance metrics on ultrasound-guided LP in pathological spine models.

Spinal curvature measurement by tracked ultrasound snapshots.

Monitoring spinal curvature in adolescent kyphoscoliosis requires regular radiographic examinations; however, the applied ionizing radiation increases the risk of cancer. Ultrasound imaging is favored over radiography because it does not emit ionizing radiation. Therefore, we tested an ultrasound system for spinal curvature measurement, with the help of spatial tracking of the ultrasound transducer. Tracked ultrasound was used to localize vertebral transverse processes as landmarks along the spine to measure curvature angles. The method was tested in two scoliotic spine models by localizing the same landmarks using both ultrasound and radiographic imaging and comparing the angles obtained. A close correlation was found between tracked ultrasound and radiographic curvature measurements. Differences between results of the two methods were 1.27 ± 0.84° (average ± SD) in an adult model and 0.96 ± 0.87° in a pediatric model. Our results suggest that tracked ultrasound may become a more tolerable and more accessible alternative to radiographic spine monitoring in adolescent kyphoscoliosis.

Open-source surface mesh-based ultrasound-guided spinal intervention simulator.

PURPOSE:    Ultrasound is prevalent in image-guided therapy as a safe, inexpensive, and widely available imaging modality. However, extensive training in interpreting ultrasound images is essential for successful procedures. An open-source ultrasound image simulator was developed to facilitate the training of ultrasound-guided spinal intervention procedures, thereby eliminating the need for an ultrasound machine from the phantom-based training environment. METHODS:    Anatomical structures and surgical tools are converted to surface meshes for data compression. Anatomical data are converted from segmented volumetric images, while the geometry of surgical tools is available as a surface mesh. The pose of the objects are either constants or coming from a pose-tracking device. Intersection points between the surface models and the ultrasound scan lines are determined with a binary space partitioning tree. The scan lines are divided into segments and filled with gray values determined by an intensity calculation accounting for material properties, reflection, and attenuation parameters defined in a configuration file. The scan lines are finally converted to a regular brightness-mode ultrasound image. RESULTS:    The simulator was tested in a tracked ultrasound imaging system, with a mock transducer tracked with an Ascension trakSTAR electromagnetic tracker, on a spine phantom. A mesh model of the spine was created from CT data. The simulated ultrasound images were generated at a speed of 50 frames per second, and a resolution of [Formula: see text] pixels, with 256 scan lines per frame, on a PC with a 3.4 GHz processor. A human subject trial was conducted to compare the learning performance of novice trainees, with real and simulated ultrasound, in the localization of facet joints of a spine phantom. With 22 participants split into two equal groups, and each participant localizing 6 facet joints, there was no statistical difference in the performance of the two groups, indicating that simulated ultrasound could indeed replace the real ultrasound in phantom-based ultrasonography training for spinal interventions. CONCLUSIONS:    The ultrasound simulator was implemented and integrated into the open-source Public Library for Ultrasound (PLUS) toolkit.