Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set II: learning curve analysis.

PURPOSE: The Johns Hopkins-Intuitive Gesture and Skill Assessment Working Set (JIGSAWS) dataset is used to develop robotic surgery skill assessment tools, but there has been no detailed analysis of this dataset. The aim of this study is to perform a learning curve analysis of the existing JIGSAWS dataset. METHODS: Five trials were performed in JIGSAWS by eight participants (four novices, two intermediates and two experts) for three exercises (suturing, knot-tying and needle passing). Global Rating Scores and time, path length and movements were analyzed quantitatively and qualitatively by graphical analysis. RESULTS: There are no significant differences in Global Rating Scale scores over time. Time in the suturing exercise and path length in needle passing had significant differences. Other kinematic parameters were not significantly different. Qualitative analysis shows a learning curve only for suturing. Cumulative sum analysis suggests completion of the learning curve for suturing by trial 4. CONCLUSIONS: The existing JIGSAWS dataset does not show a quantitative learning curve for Global Rating Scale scores, or most kinematic parameters which may be due in part to the limited size of the dataset. Qualitative analysis shows a learning curve for suturing. Cumulative sum analysis suggests completion of the suturing learning curve by trial 4. An expanded dataset is needed to facilitate subset analyses.

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software.

PURPOSE: The JIGSAWS dataset is a fixed dataset of robot-assisted surgery kinematic data used to develop predictive models of skill. The purpose of this study is to analyze the relationships of self-defined skill level with global rating scale scores and kinematic data (time, path length and movements) from three exercises (suturing, knot-tying and needle passing) (right and left hands) in the JIGSAWS dataset. METHODS: Global rating scale scores are reported in the JIGSAWS dataset and kinematic data were calculated using ROVIMAS software. Self-defined skill levels are in the dataset (novice, intermediate, expert). Correlation coefficients (global rating scale-skill level and global rating scale-kinematic parameters) were calculated. Kinematic parameters were compared among skill levels. RESULTS: Global rating scale scores correlated with skill in the knot-tying exercise (r = 0.55, p = 0.0005). In the suturing exercise, time, path length (left) and movements (left) were significantly different (p < 0.05) for novices and experts. For knot-tying, time, path length (right and left) and movements (right) differed significantly for novices and experts. For needle passing, no kinematic parameter was significantly different comparing novices and experts. The only kinematic parameter that correlated with global rating scale scores is time in the knot-tying exercise. CONCLUSION: Global rating scale scores weakly correlate with skill level and kinematic parameters. The ability of kinematic parameters to differentiate among self-defined skill levels is inconsistent. Additional data are needed to enhance the dataset and facilitate subset analyses and future model development.

An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room.

OBJECTIVE: To assess the use of a synchronized video-based motion tracking device for objective, instant, and automated assessment of laparoscopic skill in the operating room. SUMMARY BACKGROUND DATA: The assessment of technical skills is fundamental to recognition of proficient surgical practice. It is necessary to demonstrate the validity, reliability, and feasibility of any tool to be applied for objective measurement of performance. METHODS: Nineteen subjects, divided into 13 experienced (performed >100 laparoscopic cholecystectomies) and 6 inexperienced (performed <10 LCs) surgeons completed LCs on 53 patients who all had a diagnosis of biliary colic. Each procedure was recorded with the ROVIMAS video-based motion tracking device to provide an objective measure of the surgeon's dexterity. Each video was also rated by 2 experienced observers on a previously validated operative assessment scale. RESULTS: There were significant differences for motion tracking parameters between the 2 groups of surgeons for the Calot triangle dissection part of procedure for time taken (P = 0.002), total path length (P = 0.026), and number of movements (P = 0.005). Both motion tracking and video-based assessment displayed intertest reliability, and there were good correlations between the 2 modes of assessment (r = 0.4 to 0.7, P < 0.01). CONCLUSIONS: An instant, objective, valid, and reliable mode of assessment of laparoscopic performance in the operating room has been defined. This may serve to reduce the time taken for technical skills assessment, and subsequently lead to accurate and efficient audit and credentialing of surgeons for independent practice.

Curriculum-based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality.

BACKGROUND: Very few studies have addressed the transferability of skills from virtual reality (VR) to real life. The aim of this study was to assess the feasibility and effectiveness of teaching intracorporeal knot tying (ICKT) by VR simulation only. METHODS: Twenty novices underwent structured training of basic skills training on the Minimally Invasive Surgical Trainer simulator (Mentice AB, Gothenburg, Sweden) followed by knot tying training on the LapSim simulator (Surgical Science, Gothenburg, Sweden). They were assessed pre- and post-training on a video trainer. Assessment of performance included motion tracking and video-based checklist. Nonparametric statistical analysis was used, and P < .05 was deemed significant. RESULTS: All participants completed a correct knot as compared with only 25% before VR training. Time to completion was 66% faster and knot quality 45% better after VR training. Significant reduction in number of movements (P = .006) and distance traveled (P < .000) by both hands after VR training. CONCLUSIONS: Teaching ICKT by VR simulators only is feasible and effective. Furthermore, this study highlights the complementary use of different VR simulators within a structured curriculum.

Investigación en el sistema da Vinci en el hospital St Mary's de Londres / Investigation in the system da Vinci in the hospital St Mary' s of London

Hipótesis: El sistema robótico da Vinci™ permite superar las limitaciones que impone al cirujano la cirugía mínimamente invasora en términos de destreza, control del campo operatorio y ergonomía.Los estudios incluidos en esta compilación evalúan la curva de aprendizaje, la visión en tercera dimensión y el análisis de movimientos, con el propósito de comprobar las ventajas del sistema da Vinci sobre la cirugía mínimamente invasora y establecer métodos de evaluación de desempeño. Métodos: Sistema: da Vinci™. Tres estudios experimentales comparativos con diez y trece cirujanos quienes realizaron diferentes pruebas. Estudiaron: curva de aprendizaje entre cirujanos experimentados/ no experimentados, ventajas de visión 3D sobre 2D y de cirugía robótica (CR) sobre cirugía mínimamente invasora. Evaluación de mediciones: el desempeño de los cirujanos se midió con una escala global de habilidades y medición de errores. Calificación: a ciegas por expertos; además se empleó un software de análisis de movimiento para cirugía mínimamente invasora y para cirugía robótica, ambos desarrollados en el departamento y previamente validados en otros estudios. Cálculos estadísticos: programa Statistical Package for Social Sciences 10,0™. p<0,05.Resultados: La curva de aprendizaje reveló un puntaje de valoración objetiva y estructurada de destrezas técnicas de 18 (primer intento) contra 26 (quinto) p=0,02 Cronbach µ=0,894. El análisis de movimiento mostró reducción del numero de movimientos y trayectoria (p>0,01). La comparación de cirugía mínimamente invasora con cirugía robótica mostró reducción del 40 por ciento del tiempo (p=0,001) y de 70 por ciento de la trayectoria (p=0,008) con reducción de 93 por ciento de errores. La visión 3D demostró ser superior a 2D en esta y otras pruebas. La comparación entre cirugía laparoscópica y robótica en dos estudios mostró ventajas para el da Vinci.Conclusión: Los instrumentos articulados, la abolición del temblor, los movimientos a escala reducida y la visión 3D del da Vinci mejoran la habilidad y desempeño del cirujano en cirugía mínimamente invasora asistida por robot...

Synchronized video and motion analysis for the assessment of procedures in the operating theater.

HYPOTHESIS: Objective assessment of surgical skill has recently been shown to be possible through the use of dexterity-based and video analysis systems. The aim of this study was to synchronize these 2 modalities to produce a comprehensive surgical assessment tool. DESIGN: The Imperial College Surgical Assessment Device is a dexterity-based motion analysis device that has been developed in the Department of Surgical Oncology and Technology by the Surgical Computing and Imaging Research Group. Further advances to this system have been made to enable synchronized acquisition of hand kinematics and video from real procedures, and their concurrent analysis. To test the feasibility of the system, 10 laparoscopic cholecystectomies performed by 5 different surgeons on consenting patients were recorded. Analysis focused on the entire procedure and also on specific parts of the operation such as the clipping and cutting of the cystic duct and artery. RESULTS: Dexterity analysis was performed using the objective measures of time, path length, number of movements, velocities, and trajectories. Comparative analysis of a surgeon's dexterity was carried out on the whole procedure and by using the synchronized zoom facility in the software. Kinematic signals revealed rapid changes in velocity caused by alternating between different instruments or occurring after complications such as bleeding. CONCLUSION: This new motion analysis system has been shown to be an effective tool for the comprehensive assessment of operative procedures.

Laparoscopic task recognition using Hidden Markov Models.

Surgical skills assessment has been paid increased attention over the last few years. Stochastic models such as Hidden Markov Models have recently been adapted to surgery to discriminate levels of expertise. Based on our previous work combining synchronized video and motion analysis we present preliminary results of a HMM laparoscopic task recognizer which aims to model hand manipulations and to identify and recognize simple surgical tasks.

Real-time synchronization of kinematic and video data for the comprehensive assessment of surgical skills.

Surgical dexterity in operating theatres has traditionally been assessed subjectively. Electromagnetic (EM) motion tracking systems such as the Imperial College Surgical Assessment Device (ICSAD) have been shown to produce valid and accurate objective measures of surgical skill. To allow for video integration we have modified the data acquisition and built it within the ROVIMAS analysis software. We then used ActiveX 9.0 DirectShow video capturing and the system clock as a time stamp for the synchronized concurrent acquisition of kinematic data and video frames. Interactive video/motion data browsing was implemented to allow the user to concentrate on frames exhibiting certain kinematic properties that could result in operative errors. We exploited video-data synchronization to calculate the camera visual hull by identifying all 3D vertices using the ICSAD electromagnetic sensors. We also concentrated on high velocity peaks as a means of identifying potential erroneous movements to be confirmed by studying the corresponding video frames. The outcome of the study clearly shows that the kinematic data are precisely synchronized with the video frames and that the velocity peaks correspond to large and sudden excursions of the instrument tip. We validated the camera visual hull by both video and geometrical kinematic analysis and we observed that graphs containing fewer sudden velocity peaks are less likely to have erroneous movements. This work presented further developments to the well-established ICSAD dexterity analysis system. Synchronized real-time motion and video acquisition provides a comprehensive assessment solution by combining quantitative motion analysis tools and qualitative targeted video scoring.