Depth over RGB: automatic evaluation of open surgery skills using depth camera.

PURPOSE: In this paper, we present a novel approach to the automatic evaluation of open surgery skills using depth cameras. This work is intended to show that depth cameras achieve similar results to RGB cameras, which is the common method in the automatic evaluation of open surgery skills. Moreover, depth cameras offer advantages such as robustness to lighting variations, camera positioning, simplified data compression, and enhanced privacy, making them a promising alternative to RGB cameras. METHODS: Experts and novice surgeons completed two simulators of open suturing. We focused on hand and tool detection and action segmentation in suturing procedures. YOLOv8 was used for tool detection in RGB and depth videos. Furthermore, UVAST and MSTCN++ were used for action segmentation. Our study includes the collection and annotation of a dataset recorded with Azure Kinect. RESULTS: We demonstrated that using depth cameras in object detection and action segmentation achieves comparable results to RGB cameras. Furthermore, we analyzed 3D hand path length, revealing significant differences between experts and novice surgeons, emphasizing the potential of depth cameras in capturing surgical skills. We also investigated the influence of camera angles on measurement accuracy, highlighting the advantages of 3D cameras in providing a more accurate representation of hand movements. CONCLUSION: Our research contributes to advancing the field of surgical skill assessment by leveraging depth cameras for more reliable and privacy evaluations. The findings suggest that depth cameras can be valuable in assessing surgical skills and provide a foundation for future research in this area.

Automatic performance evaluation of the intracorporeal suture exercise.

PURPOSE: This work uses deep learning algorithms to provide automated feedback on the suture with intracorporeal knot exercise in the fundamentals of laparoscopic surgery simulator. Different metrics were designed to provide informative feedback to the user on how to complete the task more efficiently. The automation of the feedback will allow students to practice at any time without the supervision of experts. METHODS: Five residents and five senior surgeons participated in the study. Object detection, image classification, and semantic segmentation deep learning algorithms were used to collect statistics on the practitioner's performance. Three task-specific metrics were defined. The metrics refer to the way the practitioner holds the needle before the insertion to the Penrose drain, and the amount of movement of the Penrose drain during the needle's insertion. RESULTS: Good agreement between the human labeling and the different algorithms' performance and metric values was achieved. The difference between the scores of the senior surgeons and the surgical residents was statistically significant for one of the metrics. CONCLUSION: We developed a system that provides performance metrics of the intracorporeal suture exercise. These metrics can help surgical residents practice independently and receive informative feedback on how they entered the needle into the Penrose.