Vision-Based Tracking of Surgical Motion During Live Open-Heart Surgery.

BACKGROUND: Motion tracking during live surgeries may be used to assess surgeons' intra-operative performance, provide feedback, and predict outcome. Current assessment protocols rely on human observations, controlled laboratory settings, or tracking technologies not suitable for live operating theatres. In this study, a novel method for motion tracking of live open-heart surgery was developed, and evaluated. MATERIALS AND METHODS: Three-D-printed 'tracking die' with miniature markers were fitted to DeBakey forceps. The surgical field was recorded with a video camera mounted above the operating table. Software was developed for tracking the die from the recordings. The system was tested on five open-heart procedures. Surgeons were asked to report subjective system related concerns during live surgery and assess the weight of the die on blind test. The accuracy of the system was evaluated against ground truth generated by a robot. RESULTS: The 3D-printed die weighed 6 g and tolerated sterilization with hydrogen peroxide, which added approximately 13% to the mass of the forceps. Surgeons sensed a shift in the balance of the instrument but could on blind test not correctly verify changes in weight. When two or more markers were detected, the 3D position estimate was on average within 2-3 mm, and 1.1-2.6 degrees from ground truth. Computational time was 30-50 ms per frame on a standard laptop. CONCLUSIONS: The vision-based motion tracking system was applicable for live surgeries with negligible inconvenience to the surgeons. Motion data was extracted with acceptable accuracy and speed at low computational cost.

Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery.

Stereo 3D video from surgical procedures can be highly valuable for medical education and improve clinical communication. But access to the operating room and the surgical field is restricted. It is a sterile environment, and the physical space is crowded with surgical staff and technical equipment. In this setting, unobscured capture and realistic reproduction of the surgical procedures are difficult. This paper presents a method for rapid and reliable data collection of stereoscopic 3D videos at different camera baseline distances and distances of convergence. To collect test data with minimum interference during surgery, with high precision and repeatability, the cameras were attached to each hand of a dual-arm robot. The robot was ceiling-mounted in the operating room. It was programmed to perform a timed sequence of synchronized camera movements stepping through a range of test positions with baseline distance between 50-240 mm at incremental steps of 10 mm, and at two convergence distances of 1100 mm and 1400 mm. Surgery was paused to allow 40 consecutive 5-s video samples. A total of 10 surgical scenarios were recorded.