Computerized training system for ultrasound-guided lumbar puncture on abnormal spine models: a randomized controlled trial.

PURPOSE: A randomized controlled trial was carried out to determine whether Perk Tutor, a computerized training platform that displays an ultrasound image and real-time needle position in a three-dimensional (3D) anatomical model, would benefit residents learning ultrasound-guided lumbar puncture (LP) in simulation phantoms with abnormal spinal anatomy. METHODS: Twenty-four residents were randomly assigned to either the Perk Tutor (P) or the Control (C) group and asked to perform an LP with ultrasound guidance on part-task trainers with spinal pathology. Group P was trained with the 3D display along with conventional ultrasound imaging, while Group C used conventional ultrasound only. Both groups were then tested solely with conventional ultrasound guidance on an abnormal spinal model not previously seen. We measured potential tissue damage, needle path in tissue, total procedure time, and needle insertion time. Procedural success rate was a secondary outcome. RESULTS: The needle tracking measurements (expressed as median [interquartile range; IQR]) in Group P vs Group C revealed less potential tissue damage (39.7 [21.3-42.7] cm(2) vs 128.3 [50.3-208.2] cm(2), respectively; difference 88.6; 95% confidence intervals [CI] 24.8 to 193.5; P = 0.01), a shorter needle path inside the tissue (426.0 [164.9-571.6] mm vs 629.7 [306.4-2,879.1] mm, respectively; difference 223.7; 95% CI 76.3 to 1,859.9; P = 0.02), and lower needle insertion time (30.3 [14.0-51.0] sec vs 59.1 [26.0-136.2] sec, respectively; difference 28.8; 95% CI 2.2 to 134.0; P = 0.05). Total procedure time and overall success rates between groups did not differ. CONCLUSION: Residents trained with augmented reality 3D visualization had better performance metrics on ultrasound-guided LP in pathological spine models.

FAST skill assessment from kinematics data using convolutional neural networks.

PURPOSE: FAST is a point of care ultrasound study that evaluates for the presence of free fluid, typically hemoperitoneum in trauma patients. FAST is an essential skill for Emergency Physicians. Thus, it requires objective evaluation tools that can reduce the necessity of direct observation for proficiency assessment. In this work, we use deep neural networks to automatically assess operators' FAST skills. METHODS: We propose a deep convolutional neural network for FAST proficiency assessment based on motion data. Prior work has shown that operators demonstrate different domain-specific dexterity metrics that can distinguish novices, intermediates, and experts. Therefore, we augment our dataset with this domain knowledge and employ fine-tuning to improve the model's classification capabilities. Our model, however, does not require specific points of interest (POIs) to be defined for scanning. RESULTS: The results show that the proposed deep convolutional neural network can classify FAST proficiency with 87.5% accuracy and 0.884, 0.886, 0.247 sensitivity for novices, intermediates, and experts, respectively. It demonstrates the potential of using kinematics data as an input in FAST skill assessment tasks. We also show that the proposed domain-specific features and region fine-tuning increase the model's classification accuracy and sensitivity. CONCLUSIONS: Variations in probe motion at different learning stages can be derived from kinematics data. These variations can be used for automatic and objective skill assessment without prior identification of clinical POIs. The proposed approach can improve the quality and objectivity of FAST proficiency evaluation. Furthermore, skill assessment combining ultrasound images and kinematics data can provide a more rigorous and diversified evaluation than using ultrasound images alone.

Skills Classification in Cardiac Ultrasound with Temporal Convolution and Domain Knowledge Using a Low-Cost Probe Tracker.

As point-of-care ultrasound (POCUS) becomes more integrated into clinical practice, it is essential to address all aspects of ultrasound operator proficiency. Ultrasound proficiency requires the ability to acquire, interpret and integrate bedside ultrasound images. The difference in image acquisition psychomotor skills between novice (trainee) and expert (instructor) ultrasonographer has not been described. We created an inexpensive system, called Probe Watch, to record probe motion and assess image acquisition in cardiac POCUS using an inertial measurement device and software for data recording based on open-source components. We designed a temporal convolutional network for skills classification from probe motion that integrates clinical domain knowledge. We further designed data augmentation methods to improve its generalization. Subsequently, we validated the setup and assessment method on a set of novice and expert sonographers performing cardiac ultrasound in a simulation-based training environment. The proposed methods classified participants as novice or expert with areas under the receiver operating characteristic curve of 0.931 and 0.761 for snippets and trials, respectively. Integrating domain knowledge into the neural network had added value. Furthermore, we identified the most discriminative features for assessment. Probe Watch quantifies motion during cardiac ultrasound and provides insight into probe motion behavior. It may be deployed during cardiac ultrasound training to monitor learning curves objectively and automatically.

Reconstructing the nasal septum from instrument motion during septoplasty surgery.

Purpose: Surgery involves modifying anatomy to achieve a goal. Reconstructing anatomy can facilitate surgical care through surgical planning, real-time decision support, or anticipating outcomes. Tool motion is a rich source of data that can be used to quantify anatomy. Our work develops and validates a method for reconstructing the nasal septum from unstructured motion of the Cottle elevator during the elevation phase of septoplasty surgery, without need to explicitly delineate the surface of the septum. Approach: The proposed method uses iterative closest point registration to initially register a template septum to the tool motion. Subsequently, statistical shape modeling with iterative most likely oriented point registration is used to fit the reconstructed septum to Cottle tip position and orientation during flap elevation. Regularization of the shape model and transformation is incorporated. The proposed methods were validated on 10 septoplasty surgeries performed on cadavers by operators of varying experience level. Preoperative CT images of the cadaver septums were segmented as ground truth. Results: We estimated reconstruction error as the difference between the projections of the Cottle tip onto the surface of the reconstructed septum and the ground-truth septum segmented from the CT image. We found translational differences of 2.74 ( 2.06 - 2.81 ) mm and a rotational differences of 8.95 ( 7.11 - 10.55 ) deg between the reconstructed septum and the ground-truth septum [median (interquartile range)], given the optimal regularization parameters. Conclusions: Accurate reconstruction of the nasal septum can be achieved from tool tracking data during septoplasty surgery on cadavers. This enables understanding of the septal anatomy without need for traditional medical imaging. This result may be used to facilitate surgical planning, intraoperative care, or skills assessment.

Surgery Tutor for Computational Assessment of Technical Proficiency in Soft-Tissue Tumor Resection in a Simulated Setting.

BACKGROUND: In competency-based medical education, progression between milestones requires reliable and valid methods of assessment. Surgery Tutor is an open-source motion tracking platform developed to objectively assess technical proficiency during open soft-tissue tumor resections in a simulated setting. The objective of our study was to provide evidence in support of construct validity of the scores obtained by Surgery Tutor. We hypothesized that Surgery Tutor would discriminate between novice, intermediate, and experienced operators. METHODS: Thirty participants were assigned to novice, intermediate, or experienced groups, based on the number of prior soft-tissue resections performed. Each participant resected 2 palpable and 2 nonpalpable lesions from a soft-tissue phantom. Surgery Tutor was used to track hand and instrument motions, number of tumor breaches, and time to perform each resection. Mass of excised specimens and margin status were also recorded. RESULTS: Surgery Tutor scores demonstrated "moderate" to "good" internal structure (test-retest reliability) for novice, intermediate, and experienced groups (interclass correlation coefficient = 0.596, 0.569, 0.737; p < 0.001). Evidence in support of construct validity (consequences) was demonstrated by comparing scores of novice, intermediate, and experienced participantsfor number of hand and instrument motions (690 ± 190, 597 ± 169, 469 ± 110; p < 0.001), number of tumor breaches (29 ± 34, 16 ± 11, 9 ± 6; p < 0.001), time per resection (677 ± 331 seconds, 561 ± 210 seconds, 449 ± 148 seconds; p < 0.001), mass of completely excised specimens (22 ± 7g, 21 ± 11g, 17 ± 6 g; p = 0.035), and rate of positive margin (68%, 50%, 28%; p < 0.001). There was "strong" and "moderate" relationships between motion scores and Objective Structured Assessment of Technical Skill scores, and time per resection and Objective Structured Assessment of Technical Skill scores respectively (r = -0.60, p < 0.001; r = -0.54, p < 0.001). CONCLUSION: Surgery Tutor scores demonstrate evidenceof construct validity with regards to good internal structure, consequences, and relationship to other variables in the assessment of technical proficiency duringopen soft-tissue tumor resections in a simulated setting. Utilization of Surgery Tutor can provide formative feedback and objective assessment of surgical proficiency in a simulated setting.

Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions.

OBJECTIVE: Currently, there is a worldwide shift toward competency-based medical education. This necessitates the use of automated skills assessment methods during self-guided interventions training. Making assessment methods that are transparent and configurable will allow assessment to be interpreted into instructional feedback. The purpose of this work is to develop and validate skills assessment methods in ultrasound-guided interventions that are transparent and configurable. METHODS: We implemented a method based upon decision trees and a method based upon fuzzy inference systems for technical skills assessment. Subsequently, we validated these methods for their ability to predict scores of operators on a 25-point global rating scale in ultrasound-guided needle insertions and their ability to provide useful feedback for training. RESULTS: Decision tree and fuzzy rule-based assessment performed comparably to state-of-the-art assessment methods. They produced median errors (on a 25-point scale) of 1.7 and 1.8 for in-plane insertions and 1.5 and 3.0 for out-of-plane insertions, respectively. In addition, these methods provided feedback that was useful for trainee learning. Decision tree assessment produced feedback with median usefulness 7 out of 7; fuzzy rule-based assessment produced feedback with median usefulness 6 out of 7. CONCLUSION: Transparent and configurable assessment methods are comparable to the state of the art and, in addition, can provide useful feedback. This demonstrates their value in self-guided interventions training curricula.

Objective assessment of colonoscope manipulation skills in colonoscopy training.

OBJECTIVE: Manipulation of the colonoscope is a technical challenge for novice clinicians which is best learned in a simulated environment. It involves the coordination of scope tip steering with scope insertion, using a rotated image as reference. The purpose of this work is to develop and validate a system which objectively assesses colonoscopy technical skills proficiency in an arbitrary training environment, allowing novices to assess their technical proficiency prior to real patient encounters. METHODS: We implemented a motion tracking setup to objectively analyze and assess the way operators perform colonoscopies, including an analysis of wrist and elbow joint motions. Subsequently, we conducted a validation study to verify whether our motion analysis could discriminate novice colonoscopists from experts. Participants navigated a wooden bench-top model using a standard colonoscope while their motions were tracked. RESULTS: The developed motion tracking setup allowed colonoscopists of varying levels of proficiency to have their colonoscope manipulation assessed, and was able to be operated by a trained non-technical operator. Novice operators had significantly greater median times (101.5 vs. 31.5 s) and number of hand movements (62.0 vs. 21.5) than experts. Experts, however, spent a significantly greater proportion of time in extreme ranges of wrist and elbow joint motion than novices. CONCLUSION: We have developed and implemented a hand and joint motion analysis system that is able to discriminate novices from experts based on objective measures of motion. These metrics could, thus, serve as proxies for technical proficiency during training.

Self-guided training for deep brain stimulation planning using objective assessment.

OBJECTIVE: Deep brain stimulation (DBS) is an increasingly common treatment for neurodegenerative diseases. Neurosurgeons must have thorough procedural, anatomical, and functional knowledge to plan electrode trajectories and thus ensure treatment efficacy and patient safety. Developing this knowledge requires extensive training. We propose a training approach with objective assessment of neurosurgeon proficiency in DBS planning. METHODS: To assess proficiency, we propose analyzing both the viability of the planned trajectory and the manner in which the operator arrived at the trajectory. To improve understanding, we suggest a self-guided training course for DBS planning using real-time feedback. To validate the proposed measures of proficiency and training course, two experts and six novices followed the training course, and we monitored their proficiency measures throughout. RESULTS: At baseline, experts planned higher quality trajectories and did so more efficiently. As novices progressed through the training course, their proficiency measures increased significantly, trending toward expert measures. CONCLUSION: We developed and validated measures which reliably discriminate proficiency levels. These measures are integrated into a training course, which quantitatively improves trainee performance. The proposed training course can be used to improve trainees' proficiency, and the quantitative measures allow trainees' progress to be monitored.

Tracked ultrasonography snapshots enhance needle guidance for percutaneous renal access: a pilot study.

BACKGROUND AND PURPOSE: Although ultrasonography-guided percutaneous nephrostomy is relatively safe, a number of factors make it challenging for inexperienced operators. A computerized needle navigation technique using tracked ultrasonography snapshots was investigated to determine whether performance of percutaneous nephrostomy by inexperienced users could be improved. METHODS: Ten operators performed the procedure on a phantom model with alternating needle guidance between conventional ultrasonography and tracked ultrasonography snapshots. The needle was reinserted until fluid backflow confirmed calyceal access. Needle trajectories were recorded using the real time needle navigation system for offline evaluation of operator performance. Recorded needle trajectories were used to measure needle motion path length inside the phantom tissue, number of reinsertions, total procedure time, and needle insertion time as end points of this study. RESULTS: Needle path length measured inside the phantom tissue was significantly lower with ultrasonography snapshots guidance (295.0±23.1 mm, average±standard error of the mean) compared with control procedures (977.9±144.4 mm, P<0.01). This was associated with a significantly lower number of needle insertion attempts with ultrasonography snapshots (average 1.27±0.10 vs 2.83±0.31, P<0.01). The total procedure time and the needle insertion time were also significantly lower with ultrasonography snapshots guidance. CONCLUSION: Tracked ultrasonography snapshots appear to improve the performance of percutaneous nephrostomy in these preliminary investigations, justifying further validation studies. The presented navigation system is reproducible because of commercially available hardware and open-source software components, facilitating its potential role in clinical practice.