A Safe Framework for Quantitative In Vivo Human Evaluation of Image Guidance.

Goal: We present a new framework for in vivo image guidance evaluation and provide a case study on robotic partial nephrectomy. Methods: This framework (called the "bystander protocol") involves two surgeons, one who solely performs the therapeutic process without image guidance, and another who solely periodically collects data to evaluate image guidance. This isolates the evaluation from the therapy, so that in-development image guidance systems can be tested without risk of negatively impacting the standard of care. We provide a case study applying this protocol in clinical cases during robotic partial nephrectomy surgery. Results: The bystander protocol was performed successfully in 6 patient cases. We find average lesion centroid localization error with our IGS system to be 6.5 mm in vivo compared to our prior result of 3.0 mm in phantoms. Conclusions: The bystander protocol is a safe, effective method for testing in-development image guidance systems in human subjects.

A novel method for texture-mapping conoscopic surfaces for minimally invasive image-guided kidney surgery.

PURPOSE: Organ-level registration is critical to image-guided therapy in soft tissue. This is especially important in organs such as the kidney which can freely move. We have developed a method for registration that combines three-dimensional locations from a holographic conoscope with an endoscopically obtained textured surface. By combining these data sources clear decisions as to the tissue from which the points arise can be made. METHODS: By localizing the conoscope's laser dot in the endoscopic space, we register the textured surface to the cloud of conoscopic points. This allows the cloud of points to be filtered for only those arising from the kidney surface. Once a valid cloud is obtained we can use standard surface registration techniques to perform the image-space to physical-space registration. Since our methods use two distinct data sources we test for spatial accuracy and characterize temporal effects in phantoms, ex vivo porcine and human kidneys. In addition we use an industrial robot to provide controlled motion and positioning for characterizing temporal effects. RESULTS: Our initial surface acquisitions are hand-held. This means that we take approximately 55 s to acquire a surface. At that rate we see no temporal effects due to acquisition synchronization or probe speed. Our surface registrations were able to find applied targets with submillimeter target registration errors. CONCLUSION: The results showed that the textured surfaces could be reconstructed with submillimetric mean registration errors. While this paper focuses on kidney applications, this method could be applied to any anatomical structures where a line of sight can be created via open or minimally invasive surgical techniques.

Transorbital target localization with augmented ophthalmologic surgical endoscopy.

PURPOSE: Access to the space behind the eyeball is limited by the position of the globe anteriorly, the neurovascular structures embedded in fat posteriorly, and the tight bony confine of the orbit. These anatomical relationships have impeded application of minimally invasive procedures to the region, such as foreign body removal, tumor biopsy, or the administration of medical therapy directly to the optic nerve. An image-guided system was developed using a magnetically tracked flexible endoscope to navigate behind the eye, with the aim of enabling accurate transorbital surgery to user-specified target locations. METHODS: Targets were defined by microspherical bulbs containing water or gadolinium contrast, with differing visible coloring agent. Six living pigs were anesthetized and two microspheres of differing color and contrast content were implanted in the fat tissue of each orbit. Preoperative T1-weighted MRI volumes were obtained and registered intraoperatively. The system capabilities were tested with a series of targeted surgical interventions. The surgeon was required to navigate the endoscope to each lucent microsphere and identify it by color. For three pigs, 3D/2D registration was performed such that the target's image volume coordinates were used to display its location on real-time endoscope video. RESULTS: The ophthalmologic surgeon was able to correctly identify every target by color, with average intervention time of 24.2 min without enhancement and 3.2 min with enhancement. This difference is highly statistically significant [Formula: see text] for reduction in localization time. CONCLUSIONS: Accurate transorbital target localization is possible in-vivo using image-guided transorbital endoscopy, while endoscopic enhancement through the use of video augmentation significantly reduces procedure time.

Image-guided transorbital procedures with endoscopic video augmentation.

PURPOSE: Surgical interventions to the orbital space behind the eyeball are limited to highly invasive procedures due to the confined nature of the region along with the presence of several intricate soft tissue structures. A minimally invasive approach to orbital surgery would enable several therapeutic options, particularly new treatment protocols for optic neuropathies such as glaucoma. The authors have developed an image-guided system for the purpose of navigating a thin flexible endoscope to a specified target region behind the eyeball. Navigation within the orbit is particularly challenging despite its small volume, as the presence of fat tissue occludes the endoscopic visual field while the surgeon must constantly be aware of optic nerve position. This research investigates the impact of endoscopic video augmentation to targeted image-guided navigation in a series of anthropomorphic phantom experiments. METHODS: A group of 16 surgeons performed a target identification task within the orbits of four skull phantoms. The task consisted of identifying the correct target, indicated by the augmented video and the preoperative imaging frames, out of four possibilities. For each skull, one orbital intervention was performed with video augmentation, while the other was done with the standard image guidance technique, in random order. RESULTS: The authors measured a target identification accuracy of 95.3% and 85.9% for the augmented and standard cases, respectively, with statistically significant improvement in procedure time (Z=-2.044, p=0.041) and intraoperator mean procedure time (Z=2.456, p=0.014) when augmentation was used. CONCLUSIONS: Improvements in both target identification accuracy and interventional procedure time suggest that endoscopic video augmentation provides valuable additional orientation and trajectory information in an image-guided procedure. Utilization of video augmentation in transorbital interventions could further minimize complication risk and enhance surgeon comfort and confidence in the procedure.

Image-guided robotic surgery: update on research and potential applications in urologic surgery.

PURPOSE OF REVIEW: New methods of imaging and image-guidance technology have the potential to provide surgeons with spatially accurate three-dimensional information about the location and anatomical relationships of critical subsurface structures and instrument position updated and displayed during the performance of surgery. Robotic platforms and technology in various forms continues to revolutionize surgery and will soon incorporate image guidance. RECENT RESEARCH: Image-guided surgery (IGS) for abdominal and urologic interventions presents complex engineering and surgical challenges along with potential benefits to surgeons and patients. Key concepts such as registration, localization, accuracy, and targeting error are necessary for surgeons to understand and utilize the potential of IGS. Standard robotic surgeries, such as partial nephrectomy and radical prostatectomy may soon incorporate IGS. SUMMARY: Research continues to explore the potential for combining image guidance and robotics to augment and improve a variety of surgical interventions.

Image-Guided Abdominal Surgery and Therapy Delivery.

Image-Guided Surgery has become the standard of care in intracranial neurosurgery providing more exact resections while minimizing damage to healthy tissue. Moving that process to abdominal organs presents additional challenges in the form of image segmentation, image to physical space registration, organ motion and deformation. In this paper, we present methodologies and results for addressing these challenges in two specific organs: the liver and the kidney.

Comparison and assessment of semi-automatic image segmentation in computed tomography scans for image-guided kidney surgery.

PURPOSE: Image segmentation is integral to implementing intraoperative guidance for kidney tumor resection. Results seen in computed tomography (CT) data are affected by target organ physiology as well as by the segmentation algorithm used. This work studies variables involved in using level set methods found in the Insight Toolkit to segment kidneys from CT scans and applies the results to an image guidance setting. METHODS: A composite algorithm drawing on the strengths of multiple level set approaches was built using the Insight Toolkit. This algorithm requires image contrast state and seed points to be identified as input, and functions independently thereafter, selecting and altering method and variable choice as needed. RESULTS: Semi-automatic results were compared to expert hand segmentation results directly and by the use of the resultant surfaces for registration of intraoperative data. Direct comparison using the Dice metric showed average agreement of 0.93 between semi-automatic and hand segmentation results. Use of the segmented surfaces in closest point registration of intraoperative laser range scan data yielded average closest point distances of approximately 1 mm. Application of both inverse registration transforms from the previous step to all hand segmented image space points revealed that the distance variability introduced by registering to the semi-automatically segmented surface versus the hand segmented surface was typically less than 3 mm both near the tumor target and at distal points, including subsurface points. CONCLUSIONS: Use of the algorithm shortened user interaction time and provided results which were comparable to the gold standard of hand segmentation. Further, the use of the algorithm's resultant surfaces in image registration provided comparable transformations to surfaces produced by hand segmentation. These data support the applicability and utility of such an algorithm as part of an image guidance workflow.

Organ surface deformation measurement and analysis in open hepatic surgery: method and preliminary results from 12 clinical cases.

The incidence of soft tissue deformation has been well documented in neurosurgical procedures and is known to compromise the spatial accuracy of image-guided surgery systems.Within the context of image-guided liver surgery (IGLS), no detailed method to study and analyze the observed organ shape change between preoperative imaging and the intra-operative presentation has been developed. Contrary to the studies of deformation in neurosurgical procedures, the majority of deformation in IGLS is imposed prior to resection and due to laparotomy and mobilization. As such, methods of analyzing the organ shape change must be developed to use the intra-operative data (e.g. laser range scan (LRS) surfaces) acquired with the organ in its fully deformed shape. To achieve this end we use a signed closest point distance deformation metric computed after rigid alignment of the intra-operative LRS data with organ surfaces generated from the preoperative tomograms. The rigid alignment between the intra-operative LRS surfaces and pre-operative image data was computed with a feature weighted surface registration algorithm. In order to compare the deformation metrics across patients, an inter-patient non-rigid registration of the pre-operative CT images was performed. Given the inter-patient liver registrations, an analysis was performed to determine the potential similarities in the distribution of measured deformation between patients for which similar procedures had been performed. The results of the deformation measurement and analysis indicates the potential for soft tissue deformation to compromise surgical guidance information and suggests a similarity in imposed deformation among similar procedure types.

Kidney deformation and intraprocedural registration: a study of elements of image-guided kidney surgery.

INTRODUCTION: Central to any image-guided surgical procedure is the alignment of image and physical coordinate spaces, or registration. We explored the task of registration in the kidney through in vivo and ex vivo porcine animal models and a human study of minimally invasive kidney surgery. METHODS: A set of (n = 6) ex vivo porcine kidney models was utilized to study the effect of perfusion and loss of turgor caused by incision. Computed tomography (CT) and laser range scanner localizations of the porcine kidneys were performed before and after renal vessel clamping and after capsular incision. The da Vinci robotic surgery system was used for kidney surface acquisition and registration during robot-assisted laparoscopic partial nephrectomy. The surgeon acquired the physical surface data points with a tracked robotic instrument. These data points were aligned to preoperative CT for surface-based registrations. In addition, two biomechanical elastic computer models (isotropic and anisotropic) were constructed to simulate deformations in one of the kidneys to assess predictive capabilities. RESULTS: The mean displacement at the surface fiducials (glass beads) in six porcine kidneys was 4.4 ± 2.1 mm (range 3.4-6.7 mm), with a maximum displacement range of 6.1 to 11.2 mm. Surface-based registrations using the da Vinci robotic instrument in robot-assisted laparoscopic partial nephrectomy yielded mean and standard deviation closest point distances of 1.4 and 1.1 mm. With respect to computer model predictive capability, the target registration error was on average 6.7 mm without using the model and 3.2 mm with using the model. The maximum target error reduced from 11.4 to 6.2 mm. The anisotropic biomechanical model yielded better performance but was not statistically better. CONCLUSIONS: An initial point-based alignment followed by an iterative closest point registration is a feasible method of registering preoperative image (CT) space to intraoperative physical (robot) space. Although rigid registration provides utility for image-guidance, local deformations in regions of resection may be more significant. Computer models may be useful for prediction of such deformations, but more investigation is needed to establish the necessity of such compensation.

Towards image guided robotic surgery: multi-arm tracking through hybrid localization.

OBJECTIVE: Use of the robotic assisted surgery has been increasing in recent years, due both the continuous increase in the number of applications and the clinical benefits that surgical robots can provide. Currently robotic assisted surgery relies on endoscopic video for navigation, providing only surface visualization, thus limiting subsurface vision. To be able to visualize and identify subsurface information, techniques in image-guidance can be used. As part of designing an image guidance system, all arms of the robot need to be co-localized in a common coordinate system. METHODS: In order to track multiple arms in a common coordinate space, intrinsic and extrinsic tracking methods can be used. First, the intrinsic tracking of the daVinci, specifically of the setup joints is analyzed. Because of the inadequacy of the setup joints for co-localization a hybrid tracking method is designed and implemented to mitigate the inaccuracy of the setup joints. Different both optical and magnetic tracking methods are examined for setup joint localization. RESULTS: The hybrid localization method improved the localization accuracy of the setup joints. The inter-arm accuracy in hybrid localization was improved to 3.02 mm. This inter-arm error value was shown to be further reduced when the arms are co-registered, thus reducing common error.

Optimizing electrode placement using finite-element models in radiofrequency ablation treatment planning.

Conventional radiofrequency ablation (RFA) planning methods for identifying suitable electrode placements typically use geometric shapes to model ablation outcomes. A method is presented for searching electrode placements that couples finite-element models (FEMs) of RFA together with a novel optimization strategy. The method was designed to reduce the need for model solutions per local search step. The optimization strategy was tested against scenarios requiring single and multiple ablations. In particular, for a scenario requiring multiple ablations, a domain decomposition strategy was described to minimize the complexity of simultaneously searching multiple electrode placements. The effects of nearby vasculature on optimal electrode placement were also studied. Compared with geometric planning approaches, FEMs could potentially deliver electrode placement plans that provide more physically meaningful predictions of therapeutic outcomes.

Determining the presence of bias error using statistical methods.

Current techniques in image-guided surgery rely on the use of localizers for the measurement of position in physical space. These measurements are prone to error due to intrinsic properties of the localizer used. The error and thus accuracy of a localizer can be determined using various techniques, many of which assume that the error is isotropic and free of bias. A bias error adds an orientation dependence to the error of measured points. Determination of the presence of a bias error is an important component in the characterization of a localizer's performance. Statistical analysis of localized points on a rigid phantom can be used to detect the presence of a bias error. In this paper, we will examine the use of statistical techniques in the characterization of a series of localizers and how that information is useful in determining localizer efficacy.

Toward image guided robotic surgery: system validation.

PURPOSE: Navigation for current robotic assisted surgical techniques is primarily accomplished through a stereo pair of laparoscopic camera images. These images provide standard optical visualization of the surface but provide no subsurface information. Image guidance methods allow the visualization of subsurface information to determine the current position in relationship to that of tracked tools. MATERIALS AND METHODS: A robotic image guided surgical system was designed and implemented based on our previous laboratory studies. A series of experiments using tissue mimicking phantoms with injected target lesions was performed. The surgeon was asked to resect "tumor" tissue with and without the augmentation of image guidance using the da Vinci robotic surgical system. Resections were performed and compared to an ideal resection based on the radius of the tumor measured from preoperative computerized tomography. A quantity called the resection ratio, that is the ratio of resected tissue compared to the ideal resection, was calculated for each of 13 trials and compared. RESULTS: The mean +/- SD resection ratio of procedures augmented with image guidance was smaller than that of procedures without image guidance (3.26 +/- 1.38 vs 9.01 +/- 1.81, p <0.01). Additionally, procedures using image guidance were shorter (average 8 vs 13 minutes). CONCLUSIONS: It was demonstrated that there is a benefit from the augmentation of laparoscopic video with updated preoperative images. Incorporating our image guided system into the da Vinci robotic system improved overall tissue resection, as measured by our metric. Adding image guidance to the da Vinci robotic surgery system may result in the potential for improvements such as the decreased removal of benign tissue while maintaining an appropriate surgical margin.

Feasibility study for image-guided kidney surgery: assessment of required intraoperative surface for accurate physical to image space registrations.

A notable complication of applying current image-guided surgery techniques of soft tissue to kidney resections (nephrectomies) is the limited field of view of the intraoperative kidney surface. This limited view constrains the ability to obtain a sufficiently geometrically descriptive surface for accurate surface-based registrations. The authors examined the effects of the limited view by using two orientations of a kidney phantom to model typical laparoscopic and open partial nephrectomy views. Point-based registrations, using either rigidly attached markers or anatomical landmarks as fiducials, served as initial alignments for surface-based registrations. Laser range scanner (LRS) obtained surfaces were registered to the phantom's image surface using a rigid iterative closest point algorithm. Subsets of each orientation's LRS surface were used in a robustness test to determine which parts of the surface yield the most accurate registrations. Results suggest that obtaining accurate registrations is a function of the percentage of the total surface and of geometric surface properties, such as curvature. Approximately 28% of the total surface is required regardless of the location of that surface subset. However, that percentage decreases when the surface subset contains information from opposite ends of the surface and/or unique anatomical features, such as the renal artery and vein.

Robust surface registration using salient anatomical features for image-guided liver surgery: algorithm and validation.

A successful surface-based image-to-physical space registration in image-guided liver surgery (IGLS) is critical to provide reliable guidance information to surgeons and pertinent surface displacement data for use in deformation correction algorithms. The current protocol used to perform the image-to-physical space registration involves an initial pose estimation provided by a point based registration of anatomical landmarks identifiable in both the preoperative tomograms and the intraoperative presentation. The surface based registration is then performed via a traditional iterative closest point (ICP) algorithm between the preoperative liver surface, segmented from the tomographic image set, and an intraoperatively acquired point cloud of the liver surface provided by a laser range scanner. Using this more conventional method, the registration accuracy can be compromised by poor initial pose estimation as well as tissue deformation due to the laparotomy and liver mobilization performed prior to tumor resection. In order to increase the robustness of the current surface-based registration method used in IGLS, we propose the incorporation of salient anatomical features, identifiable in both the preoperative image sets and intraoperative liver surface data, to aid in the initial pose estimation and play a more significant role in the surface-based registration via a novel weighting scheme. Examples of such salient anatomical features are the falciform groove region as well as the inferior ridge of the liver surface. In order to validate the proposed weighted patch registration method, the alignment results provided by the proposed algorithm using both single and multiple patch regions were compared with the traditional ICP method using six clinical datasets. Robustness studies were also performed using both phantom and clinical data to compare the resulting registrations provided by the proposed algorithm and the traditional method under conditions of varying initial pose. The results provided by the robustness trials and clinical registration comparisons suggest that the proposed weighted patch registration algorithm provides a more robust method with which to perform the image-to-physical space registration in IGLS. Furthermore, the implementation of the proposed algorithm during surgical procedures does not impose significant increases in computation or data acquisition times.

Characterization of tracked radiofrequency ablation in phantom.

In radiofrequency ablation (RFA), successful therapy requires accurate, image-guided placement of the ablation device in a location selected by a predictive treatment plan. Current planning methods rely on geometric models of ablations that are not sensitive to underlying physical processes in RFA. Implementing plans based on computational models of RFA with image-guided techniques, however, has not been well characterized. To study the use of computational models of RFA in planning needle placement, this work compared ablations performed with an optically tracked RFA device with corresponding models of the ablations. The calibration of the tracked device allowed the positions of distal features of the device, particularly the tips of the needle electrodes, to be determined to within 1.4 +/- 0.6 mm of uncertainty. Ablations were then performed using the tracked device in a phantom system based on an agarose-albumin mixture. Images of the sliced phantom obtained from the ablation experiments were then compared with the predictions of a bioheat transfer model of RFA, which used the positional data of the tracked device obtained during ablation. The model was demonstrated to predict 90% of imaged pixels classified as being ablated. The discrepancies between model predictions and observations were analyzed and attributed to needle tracking inaccuracy as well as to uncertainties in model parameters. The results suggest the feasibility of using finite element modeling to plan ablations with predictable outcomes when implemented using tracked RFA.

Laparoscopic surface scanning and subsurface targeting: implications for image-guided laparoscopic liver surgery.

Segmental liver resection and locoregional ablative therapies are dependent upon accurate tumor localization to ensure safety as well as acceptable oncologic results. Because of the liver's limited external landmarks and complex internal anatomy, such tumor localization poses a technical challenge. Image guided therapies (IGT) address this problem by mapping the real-time, intraoperative position of surgical instruments onto preoperative tomographic imaging through a process called registration. Accuracy is critical to IGT and is a function of: 1) the registration technique, 2) the tissue characteristics, and 3) imaging techniques. The purpose of this study is to validate a novel method of registration using an endoscopic Laser Range Scanner (eLRS) and demonstrate its applicability to laparoscopic liver surgery. Six radiopaque targets were inserted into an ex-vivo bovine liver and a computed tomography (CT) scan was obtained. Using the eLRS, the liver surface was scanned and a surface-based registration was constructed to predict the position of the intraparenchymal targets. The target registration error (TRE) achieved using our surface-based registration was 2.4 +/- 1.0 mm. A comparable TRE using traditional fiducial-based registration was 2.6 +/- 1.7 mm. Compared to traditional fiducial-based registration, laparoscopic surface scanning is able to predict the location of intraparenchymal liver targets with similar accuracy and rate of data acquisition.

Concepts and preliminary data toward the realization of image-guided liver surgery.

Image-guided surgery provides navigational assistance to the surgeon by displaying the surgical probe position on a set of preoperative tomograms in real time. In this study, the feasibility of implementing image-guided surgery concepts into liver surgery was examined during eight hepatic resection procedures. Preoperative tomographic image data were acquired and processed. Accompanying intraoperative data on liver shape and position were obtained through optically tracked probes and laser range scanning technology. The preoperative and intraoperative representations of the liver surface were aligned using the iterative closest point surface matching algorithm. Surface registrations resulted in mean residual errors from 2 to 6 mm, with errors of target surface regions being below a stated goal of 1 cm. Issues affecting registration accuracy include liver motion due to respiration, the quality of the intraoperative surface data, and intraoperative organ deformation. Respiratory motion was quantified during the procedures as cyclical, primarily along the cranial-caudal direction. The resulting registrations were more robust and accurate when using laser range scanning to rapidly acquire thousands of points on the liver surface and when capturing unique geometric regions on the liver surface, such as the inferior edge. Finally, finite element models recovered much of the observed intraoperative deformation, further decreasing errors in the registration. Image-guided liver surgery has shown the potential to provide surgeons with important navigation aids that could increase the accuracy of targeting lesions and the number of patients eligible for surgical resection.

Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements.

Image-guided liver surgery requires the ability to identify and compensate for soft tissue deformation in the organ. The predeformed state is represented as a complete three-dimensional surface of the organ, while the intraoperative data is a range scan point cloud acquired from the exposed liver surface. The first step is to rigidly align the coordinate systems of the intraoperative and preoperative data. Most traditional rigid registration methods minimize an error metric over the entire data set. In this paper, a new deformation-identifying rigid registration (DIRR) is reported that identifies and aligns minimally deformed regions of the data using a modified closest point distance cost function. Once a rigid alignment has been established, deformation is accounted for using a linearly elastic finite element model (FEM) and implemented using an incremental framework to resolve geometric nonlinearities. Boundary conditions for the incremental formulation are generated from intraoperatively acquired range scan surfaces of the exposed liver surface. A series of phantom experiments is presented to assess the fidelity of the DIRR and the combined DIRR/FEM approaches separately. The DIRR approach identified deforming regions in 90% of cases under conditions of realistic surgical exposure. With respect to the DIRR/FEM algorithm, subsurface target errors were correctly located to within 4 mm in phantom experiments.

Development of a three-dimensional freehand endorectal ultrasound system for use in rectal cancer imaging.

The initial study reporting the accuracy of an optically tracked endorectal ultrasound (TERUS) probe for the purpose of improving the staging of rectal cancer is presented here. In this work we describe the need for a more accurate ERUS system and why the incorporation of image guidance makes this goal feasible. A rectal phantom was constructed with five targets placed in positions where tumors normally occur. The locations of these targets were found using two different imaging modalities, CT and ultrasound, and the target registration error (TRE) between these two image sets was calculated. The average TRE of 33 image captures of the five targets using TERUS was 2.1 mm. This is a promising outcome because the desired tumor margins for rectal cancer are on the order of centimeters. These preliminary results support the proof of concept for a TERUS system that should improve ultrasound imaging in rectal cancer.