A holographic telementoring system depicting surgical instrument movements for real-time guidance in open surgeries.

BACKGROUND AND OBJECTIVE: During open surgeries, telementoring serves as a valuable tool for transferring surgical knowledge from a specialist surgeon (mentor) to an operating surgeon (mentee). Depicting the intended movements of the surgical instruments over the operative field improves the understanding of the required tool-tissue interaction. The objective of this work is to develop a telementoring system tailored for open surgeries, enabling the mentor to remotely demonstrate the necessary motions of surgical instruments to the mentee. METHODS: A remote telementoring system for open surgery was implemented. The system generates visual cues in the form of virtual surgical instrument motion augmented onto the live view of the operative field. These cues can be rendered on both conventional screens in the operating room and as dynamic holograms on a head mounted display device worn by the mentee. The technical performance of the system was evaluated, where the operating room and remote location were geographically separated and connected via the Internet. Additionally, user studies were conducted to assess the effectiveness of the system as a mentoring tool. RESULTS: The system took 307 ± 12 ms to transmit an operative field view of 1920 × 1080 resolution, along with depth information spanning 36 cm, from the operating room to the remote location. Conversely, it took 145 ± 14 ms to receive the motion of virtual surgical instruments from the remote location back to the operating room. Furthermore, the user studies demonstrated: (a) mentor's capability to annotate the operative field with an accuracy of 3.92 ± 2.1 mm, (b) mentee's ability to comprehend and replicate the motion of surgical instruments in real-time with an average deviation of 12.8 ± 3 mm, (c) efficacy of the rendered dynamic holograms in conveying information intended for surgical instrument motion. CONCLUSIONS: The study demonstrates the feasibility of transmitting information over the Internet from the mentor to the mentee in the form of virtual surgical instruments' motion and projecting it as holograms onto the live view of the operative field. This holds potential to enhance real-time collaborative capabilities between the mentor and the mentee during an open surgery.

An Actuated Variable-View Rigid Scope System to Assist Visualization in Diagnostic Procedures.

OBJECTIVE: Variable-view rigid scopes offer advantages compared to traditional angled laparoscopes for examining a diagnostic site. However, altering the scope's view requires a high level of dexterity and understanding of spatial orientation. This requires an intuitive mechanism to allow an operator to easily understand the anatomical surroundings and smoothly adjust the scope's focus during diagnosis. To address this challenge, the objective of this work is to develop a mechanized arm that assists in visualization using variable-view rigid scopes during diagnostic procedures. METHODS: A system with a mechanized arm to maneuver a variable-view rigid scope (EndoCAMeleon - Karl Storz) was developed. A user study was conducted to assess the ability of the proposed mechanized arm for diagnosis in a preclinical navigation task and a simulated cystoscopy procedure. RESULTS: The mechanized arm performed significantly better than direct maneuvering of the rigid scope. In the preclinical navigation task, it reduced the percentage of time the scope's focus shifted outside a predefined track. Similarly, for simulated cystoscopy procedure, it reduced the duration and the perceived workload. CONCLUSION: The proposed mechanized arm enhances the operator's ability to accurately maneuver a variable-view rigid scope and reduces the effort in performing diagnostic procedures.Clinical and Translational Impact Statement: The preclinical research introduces a mechanized arm to intuitively maneuver a variable-view rigid scope during diagnostic procedures, while minimizing the mental and physical workload to the operator.

A generic scope actuation system for flexible endoscopes.

BACKGROUND: A scope actuation system assists a surgeon in steering a scope for navigating an operative field during an interventional or diagnostic procedure. Each system is tailored for a specific surgical procedure. The development of a generic scope actuation system could assist various laparoscopic and endoscopic procedures. This has the potential to reduce the deployment and maintenance costs for a hospital, making it more accessible for clinical usage. METHODS: A modular actuation system (for maneuvering rigid laparoscopes) was adapted to enable incorporation of flexible endoscopes. The design simplifies the installation and disassembly processes. User studies were conducted to assess the ability of the system to focus onto a diagnostic area, and to navigate during a simulated esophagogastroduodenoscopy procedure. During the studies, the endoscope was maneuvered with (robotic mode) and without (manual mode) the actuation system to navigate the endoscope's focus on a predefined track. RESULTS: Results show that the robotic mode performed better than the manual mode on all the measured performance parameters including (a) the total duration to traverse a track, (b) the percentage of time spent outside a track while traversing, and (c) the number of times the scope focus shifts outside the track. Additionally, robotic mode also reduced the perceived workload based on the NASA-TLX scale. CONCLUSIONS: The proposed scope actuation system enhances the maneuverability of flexible endoscopes. It also lays the groundwork for future development of modular and generic scope assistant systems that can be used in both laparoscopic and endoscopic procedures.

Development and Evaluation of a Mixed-Reality Tele-ultrasound System.

OBJECTIVE: The objective of this feasibility study was to develop and assess a tele-ultrasound system that would enable an expert sonographer (situated at the remote site) to provide real-time guidance to an operator (situated at the imaging site) using a mixed-reality environment. METHODS: An architecture along with the operational workflow of the system is designed and a prototype is developed that enables guidance in form of audiovisual cues. The visual cues comprise holograms (of the ultrasound images and ultrasound probe) and is rendered to the operator using a head-mounted display device. The position and orientation of the ultrasound probe's hologram are remotely controlled by the expert sonographer and guide the placement of a physical ultrasound probe at the imaging site. The developed prototype was evaluated for its performance on a network. In addition, a user study (with 12 participants) was conducted to assess the operator's ability to align the probe under different guidance modes. RESULTS: The network performance revealed the view of the imaging site and ultrasound images were transferred to the remote site in 233 ± 42 and 158 ± 38 ms, respectively. The expert sonographer was able to transfer, to the imaging site, data related to position and orientation of the ultrasound probe's hologram in 78 ± 13 ms. The user study indicated that the audiovisual cues are sufficient for an operator to position and orient a physical probe for accurate depiction of the targeted tissue (p < 0.001). The probe's placement translational and rotational errors were 1.4 ± 0.6 mm and 5.4 ± 2.2º. CONCLUSION: The work illustrates the feasibility of using a mixed-reality environment for effective communication between an expert sonographer (ultrasound physician) and an operator. Further studies are required to determine its applicability in a clinical setting during tele-ultrasound.

Scope actuation system for articulated laparoscopes.

BACKGROUND: An articulated laparoscope comprises a rigid shaft with an articulated distal end to change the viewing direction. The articulation provides improved navigation of the operating field in confined spaces. Furthermore, incorporation of an actuation system tends to enhance the control of an articulated laparoscope. METHODS: A preliminary prototype of a scope actuation system to maneuver an off-the-shelf articulated laparoscope (EndoCAMaleon by Karl Storz, Germany) was developed. A user study was conducted to evaluate this prototype for the surgical paradigm of video-assisted thoracic surgery. In the study, the subjects maneuvered an articulated scope under two modes of operation: (a) actuated mode where an operating surgeon maneuvers the scope using the developed prototype and (b) manual mode where a surgical assistant directly maneuvers the scope. The actuated mode was further assessed for multiple configurations based on the orientation of the articulated scope at the incision. RESULTS: The data show the actuated mode scored better than the manual mode on all the measured performance parameters including (a) total duration to visualize a marked region, (a) duration for which scope focus shifts outside a predefined visualization region, and (c) number of times for which scope focus shifts outside a predefined visualization region. Among the different configurations tested using the actuated mode, no significant difference was observed. CONCLUSIONS: The proposed articulated scope actuation system facilitates better navigation of an operative field as compared to a human assistant. Secondly, irrespective of the orientation in which an articulated scope's shaft is inserted through an incision, the proposed actuation system can navigate and visualize the operative field.

Preliminary design and evaluation of a generic surgical scope adapter.

BACKGROUND: Robotic scope assistant systems are used to visualise and navigate the operative field during a laparoscopic surgery. The objective of this work is to design a surgical scope adapter that enables control of different scope types (zero-degree, angulated, and articulated), and can be connected to any six degree-of-freedom robotic manipulator for usage as a robotic scope assistant system. METHODS: A surgical scope adapter compatible with different camera heads and scope types was designed and prototyped. The technical performance of the scope adapter was evaluated and a user study was conducted to assess the human-in-the-loop control. RESULTS: All the subjects were able to navigate the simulated operative field. The scope adapter permits continuous motion to explore the operative field as well as intermittent motion to accurately focus on the targeted anatomical landmarks. CONCLUSION: The modular and generic nature of the surgical scope adapter may enable its usage across different minimally invasive surgeries.

Evaluation of user-interfaces for controlling movements of virtual minimally invasive surgical instruments.

BACKGROUND: Recent tele-mentoring technologies for minimally invasive surgery (MIS) augments the operative field with movements of virtual surgical instruments as visual cues. The objective of this work is to assess different user-interfaces that effectively transfer mentor's hand gestures to the movements of virtual surgical instruments. METHODS: A user study was conducted to assess three different user-interface devices (Oculus-Rift, SpaceMouse, Touch Haptic device) under various scenarios. The devices were integrated with a MIS tele-mentoring framework for control of both manual and robotic virtual surgical instruments. RESULTS: The user study revealed that Oculus Rift is preferred during robotic scenarios, whereas the touch haptic device is more suitable during manual scenarios for tele-mentoring. CONCLUSION: A user-interface device in the form of a stylus controlled by fingers for pointing in 3D space is more suitable for manual MIS, whereas a user-interface that can be moved and oriented easily in 3D space by wrist motion is more suitable for robotic MIS.

Preliminary design and evaluation of a remote tele-mentoring system for minimally invasive surgery.

BACKGROUND: Tele-mentoring during surgery facilitates the transfer of surgical knowledge from a mentor (specialist surgeon) to a mentee (operating surgeon). The aim of this work is to develop a tele-mentoring system tailored for minimally invasive surgery (MIS) where the mentor can remotely demonstrate to the mentee the required motion of the surgical instruments. METHODS: A remote tele-mentoring system is implemented that generates visual cues in the form of virtual surgical instrument motion overlaid onto the live view of the operative field. The technical performance of the system is evaluated in a simulated environment, where the operating room and the central location of the mentor were physically located in different countries and connected over the internet. In addition, a user study was performed to assess the system as a mentoring tool. RESULTS: On average, it took 260 ms to send a view of the operative field of 1920 × 1080 resolution from the operating room to the central location of the mentor and an average of 132 ms to receive the motion of virtual surgical instruments from the central location to the operating room. The user study showed that it is feasible for the mentor to demonstrate and for the mentee to understand and replicate the motion of surgical instruments. CONCLUSION: The work demonstrates the feasibility of transferring information over the internet from a mentor to a mentee in the form of virtual surgical instruments. Their motion is overlaid onto the live view of the operative field enabling real-time interactions between both the surgeons.

Towards development of a tele-mentoring framework for minimally invasive surgeries.

BACKGROUND: Tele-mentoring facilitates the transfer of surgical knowledge. The objective of this work is to develop a tele-mentoring framework that enables a specialist surgeon to mentor an operating surgeon by transferring information in a form of surgical instruments' motion required during a minimally invasive surgery. METHOD: A tele-mentoring framework is developed to transfer video stream of the surgical field, poses of the scope and port placement from the operating room to a remote location. From the remote location, the motion of virtual surgical instruments augmented onto the surgical field is sent to the operating room. RESULTS: The proposed framework is suitable to be integrated with laparoscopic as well as robotic surgeries. It takes on average 1.56 s to send information from the operating room to the remote location and 0.089 s for vice versa over a local area network. CONCLUSIONS: The work demonstrates a tele-mentoring framework that enables a specialist surgeon to mentor an operating surgeon during a minimally invasive surgery.