Surgical Tattoos in Infrared: A Dataset for Quantifying Tissue Tracking and Mapping.

Quantifying performance of methods for tracking and mapping tissue in endoscopic environments is essential for enabling image guidance and automation of medical interventions and surgery. Datasets developed so far either use rigid environments, visible markers, or require annotators to label salient points in videos after collection. These are respectively: not general, visible to algorithms, or costly and error-prone. We introduce a novel labeling methodology along with a dataset that uses said methodology, Surgical Tattoos in Infrared (STIR). STIR has labels that are persistent but invisible to visible spectrum algorithms. This is done by labelling tissue points with IR-fluorescent dye, indocyanine green (ICG), and then collecting visible light video clips. STIR comprises hundreds of stereo video clips in both in vivo and ex vivo scenes with start and end points labelled in the IR spectrum. With over 3,000 labelled points, STIR will help to quantify and enable better analysis of tracking and mapping methods. After introducing STIR, we analyze multiple different frame-based tracking methods on STIR using both 3D and 2D endpoint error and accuracy metrics. STIR is available at https://dx.doi.org/10.21227/w8g4-g548.

MRI-Compatible Pneumatic Robot for Transperineal Prostate Needle Placement.

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of prostate and surrounding tissue, thus granting potential to be a superior medical imaging modality for guiding and monitoring prostatic interventions. However, the benefits cannot be readily harnessed for interventional procedures due to difficulties that surround the use of high-field (1.5T or greater) MRI. The inability to use conventional mechatronics and the confined physical space makes it extremely challenging to access the patient. We have designed a robotic assistant system that overcomes these difficulties and promises safe and reliable intraprostatic needle placement inside closed high-field MRI scanners. MRI compatibility of the robot has been evaluated under 3T MRI using standard prostate imaging sequences and average SNR loss is limited to 5%. Needle alignment accuracy of the robot under servo pneumatic control is better than 0.94 mm rms per axis. The complete system workflow has been evaluated in phantom studies with accurate visualization and targeting of five out of five 1 cm targets. The paper explains the robot mechanism and controller design, the system integration, and presents results of preliminary evaluation of the system.

3D simulation of needle-tissue interaction with application to prostate brachytherapy.

This paper presents a needle-tissue interaction model that is a 3D extension of prior work based on needle and tissue models discretized using the Finite Element Method. The use of flexible needles necessitates remeshing the tissue during insertion, since simple mesh-node snapping to the tip can be detrimental to the simulation. In this paper, node repositioning and node addition are the two methods of mesh modification examined for coarse meshes. Our focus is on numerical approaches for fast implementation of these techniques. Although the two approaches compared, namely the Woodbury formula (matrix inversion lemma) and the boundary condition switches, have the same computational complexity, the Woodbury formula is shown to perform faster due to its cache-efficient order of operations. Furthermore, node addition is applied in constant time for both approaches, whereas node repositioning requires longer and variable computational times. A method for rendering the needle forces during simulated insertions into a 3D prostate model has been implemented. Combined with a detailed anatomical segmentation, this will be useful in teaching the practice of prostate brachytherapy. Issues related to discretization of such coupled (e.g., needle-tissue) models are also discussed.

Interactive simulation of needle insertion models.

A novel interactive virtual needle insertion simulation is presented. The simulation models are based on measured planar tissue deformations and needle insertion forces. Since the force-displacement relationship is only of interest along the needle shaft, a condensation technique is shown to reduce the computational complexity of linear simulation models significantly. As the needle penetrates or is withdrawn from the tissue model, the boundary conditions that determine the tissue and needle motion change. Boundary condition and local material coordinate changes are facilitated by fast low-rank matrix updates. A large-strain elastic needle model is coupled to the tissue models to account for needle deflection and bending during simulated insertion. A haptic environment, based on these novel interactive simulation techniques, allows users to manipulate a three-degree-of-freedom virtual needle as it penetrates virtual tissue models, while experiencing steering torques and lateral needle forces through a planar haptic interface.

Needle steering and motion planning in soft tissues.

In this work, needle insertion into deformable tissue is formulated as a trajectory planning and control problem. A new concept of needle steering has been developed and a needle manipulation Jacobian defined using numerical needle insertion models that include needle deflection and soft tissue deformation. This concept is used in conjunction with a potential-field-based path planning technique to demonstrate needle tip placement and obstacle avoidance. Results from open loop insertion experiments are provided.

Robot-assisted laparoscopic ultrasonography for hepatic surgery.

INTRODUCTION: This study describes and evaluates a novel, robot-assisted laparoscopic ultrasonographic device for hepatic surgery. Laparoscopic liver surgery is being performed with increasing frequency. One major drawback of this approach is the limited capability of intraoperative ultrasonography (IOUS) using standard laparoscopic devices. Robotic surgery systems offer the opportunity to develop new tools to improve techniques in minimally invasive surgery. This study evaluates a new integrated ultrasonography (US) device with the da Vinci Surgical System for laparoscopic visualization, comparing it with conventional handheld laparoscopic IOUS for performing key tasks in hepatic surgery. METHODS: A prototype laparoscopic IOUS instrument was developed for the da Vinci Surgical System and compared with a conventional laparoscopic US device in simulation tasks: (1) In vivo porcine hepatic visualization and probe manipulation, (2) lesion detection accuracy, and (3) biopsy precision. Usability was queried by poststudy questionnaire. RESULTS: The robotic US proved better than conventional laparoscopic US in liver surface exploration (85% success vs 73%; P = .030) and tool manipulation (79% vs 57%; P = .028), whereas no difference was detected in lesion identification (63 vs 58; P = .41) and needle biopsy tasks (57 vs 48; P = .11). Subjects found the robotic US to facilitate better probe positioning (80%), decrease fatigue (90%), and be more useful overall (90%) on the post-task questionnaire. CONCLUSION: We found this robot-assisted IOUS system to be practical and useful in the performance of important tasks required for hepatic surgery, outperforming free-hand laparoscopic IOUS for certain tasks, and was more subjectively usable to the surgeon. Systems such as this may expand the use of robotic surgery for complex operative procedures requiring IOUS.

Integrated navigation and control software system for MRI-guided robotic prostate interventions.

A software system to provide intuitive navigation for MRI-guided robotic transperineal prostate therapy is presented. In the system, the robot control unit, the MRI scanner, and the open-source navigation software are connected together via Ethernet to exchange commands, coordinates, and images using an open network communication protocol, OpenIGTLink. The system has six states called "workphases" that provide the necessary synchronization of all components during each stage of the clinical workflow, and the user interface guides the operator linearly through these workphases. On top of this framework, the software provides the following features for needle guidance: interactive target planning; 3D image visualization with current needle position; treatment monitoring through real-time MR images of needle trajectories in the prostate. These features are supported by calibration of robot and image coordinates by fiducial-based registration. Performance tests show that the registration error of the system was 2.6mm within the prostate volume. Registered real-time 2D images were displayed 1.97 s after the image location is specified.

Software strategy for robotic transperineal prostate therapy in closed-bore MRI.

A software strategy to provide intuitive navigation for MRI-guided robotic transperineal prostate therapy is presented. In the system, the robot control unit, the MRI scanner, and open-source navigation software are connected to one another via Ethernet to exchange commands, coordinates, and images. Six states of the system called "workphases" are defined based on the clinical scenario to synchronize behaviors of all components. The wizard-style user interface allows easy following of the clinical workflow. On top of this framework, the software provides features for intuitive needle guidance: interactive target planning; 3D image visualization with current needle position; treatment monitoring through real-time MRI. These features are supported by calibration of robot and image coordinates by the fiducial-based registration. The performance test shows that the registration error of the system was 2.6 mm in the prostate area, and it displayed real-time 2D image 1.7 s after the completion of image acquisition.

Compensation of geometric distortion effects on intraoperative magnetic resonance imaging for enhanced visualization in image-guided neurosurgery.

OBJECTIVE: Preoperative magnetic resonance imaging (MRI), functional MRI, diffusion tensor MRI, magnetic resonance spectroscopy, and positron-emission tomographic scans may be aligned to intraoperative MRI to enhance visualization and navigation during image-guided neurosurgery. However, several effects (both machine- and patient-induced distortions) lead to significant geometric distortion of intraoperative MRI. Therefore, a precise alignment of these image modalities requires correction of the geometric distortion. We propose and evaluate a novel method to compensate for the geometric distortion of intraoperative 0.5-T MRI in image-guided neurosurgery. METHODS: In this initial pilot study, 11 neurosurgical procedures were prospectively enrolled. The scheme used to correct the geometric distortion is based on a nonrigid registration algorithm introduced by our group. This registration scheme uses image features to establish correspondence between images. It estimates a smooth geometric distortion compensation field by regularizing the displacements estimated at the correspondences. A patient-specific linear elastic material model is used to achieve the regularization. The geometry of intraoperative images (0.5 T) is changed so that the images match the preoperative MRI scans (3 T). RESULTS: We compared the alignment between preoperative and intraoperative imaging using 1) only rigid registration without correction of the geometric distortion, and 2) rigid registration and compensation for the geometric distortion. We evaluated the success of the geometric distortion correction algorithm by measuring the Hausdorff distance between boundaries in the 3-T and 0.5-T MRIs after rigid registration alone and with the addition of geometric distortion correction of the 0.5-T MRI. Overall, the mean magnitude of the geometric distortion measured on the intraoperative images is 10.3 mm with a minimum of 2.91 mm and a maximum of 21.5 mm. The measured accuracy of the geometric distortion compensation algorithm is 1.93 mm. There is a statistically significant difference between the accuracy of the alignment of preoperative and intraoperative images, both with and without the correction of geometric distortion (P < 0.001). CONCLUSION: The major contributions of this study are 1) identification of geometric distortion of intraoperative images relative to preoperative images, 2) measurement of the geometric distortion, 3) application of nonrigid registration to compensate for geometric distortion during neurosurgery, 4) measurement of residual distortion after geometric distortion correction, and 5) phantom study to quantify geometric distortion.

Robotic assistant for transperineal prostate interventions in 3T closed MRI.

Numerous studies have demonstrated the efficacy of image-guided needle-based therapy and biopsy in the management of prostate cancer. The accuracy of traditional prostate interventions performed using transrectal ultrasound (TRUS) is limited by image fidelity, needle template guides, needle deflection and tissue deformation. Magnetic Resonance Imaging (MRI) is an ideal modality for guiding and monitoring such interventions due to its excellent visualization of the prostate, its sub-structure and surrounding tissues. We have designed a comprehensive robotic assistant system that allows prostate biopsy and brachytherapy procedures to be performed entirely inside a 3T closed MRI scanner. We present a detailed design of the robotic manipulator and an evaluation of its usability and MR compatibility.

3D needle-tissue interaction simulation for prostate brachytherapy.

This paper presents a needle-tissue interaction model that is a 3D extension of a prior work based on the finite element method. The model is also adapted to accommodate arbitrary meshes so that the anatomy can effectively be meshed using third-party algorithms. Using this model a prostate brachytherapy simulator is designed to help medical residents acquire needle steering skills. This simulation uses a prostate mesh generated from clinical data segmented as contours on parallel slices. Node repositioning and addition, which are methods for achieving needle-tissue coupling, are discussed. In order to achieve realtime haptic rates, computational approaches to these methods are compared. Specifically, the benefit of using the Woodbury formula (matrix inversion lemma) is studied. Our simulation of needle insertion into a prostate is shown to run faster than 1 kHz.