Predicting target vessel location on robot-assisted coronary artery bypass graft using CT to ultrasound registration.

PURPOSE: Although robot-assisted coronary artery bypass grafting (RA-CABG) has gained more acceptance worldwide, its success still depends on the surgeon's experience and expertise, and the conversion rate to full sternotomy is in the order of 15%-25%. One of the reasons for conversion is poor pre-operative planning, which is based solely on pre-operative computed tomography (CT) images. In this paper, the authors propose a technique to estimate the global peri-operative displacement of the heart and to predict the intra-operative target vessel location, validated via both an in vitro and a clinical study. METHODS: As the peri-operative heart migration during RA-CABG has never been reported in the literatures, a simple in vitro validation study was conducted using a heart phantom. To mimic the clinical workflow, a pre-operative CT as well as peri-operative ultrasound images at three different stages in the procedure (Stage(0)-following intubation; Stage(1)-following lung deflation; and Stage(2)-following thoracic insufflation) were acquired during the experiment. Following image acquisition, a rigid-body registration using iterative closest point algorithm with the robust estimator was employed to map the pre-operative stage to each of the peri-operative ones, to estimate the heart migration and predict the peri-operative target vessel location. Moreover, a clinical validation of this technique was conducted using offline patient data, where a Monte Carlo simulation was used to overcome the limitations arising due to the invisibility of the target vessel in the peri-operative ultrasound images. RESULTS: For the in vitro study, the computed target registration error (TRE) at Stage(0), Stage(1), and Stage(2) was 2.1, 3.3, and 2.6 mm, respectively. According to the offline clinical validation study, the maximum TRE at the left anterior descending (LAD) coronary artery was 4.1 mm at Stage(0), 5.1 mm at Stage(1), and 3.4 mm at Stage(2). CONCLUSIONS: The authors proposed a method to measure and validate peri-operative shifts of the heart during RA-CABG. In vitro and clinical validation studies were conducted and yielded a TRE in the order of 5 mm for all cases. As the desired clinical accuracy imposed by this procedure is on the order of one intercostal space (10-15 mm), our technique suits the clinical requirements. The authors therefore believe this technique has the potential to improve the pre-operative planning by updating peri-operative migration patterns of the heart and, consequently, will lead to reduced conversion to conventional open thoracic procedures.

Surgeon-controlled visualization techniques for virtual reality-guided cardiac surgery.

Surgeon-to-computer interaction difficulties in using virtual reality (VR)-guided surgical environments arise from disorientation, insufficient depth information, and delegation of view control. This study focuses on optimizing information delivery for VR-guided beating heart surgery. Initial human factors evaluation and participatory design has provided insight for developing an effective surgeon-controlled interface for VR-guided cardiac interventions. We discuss the motivation for and development of three interface prototypes as well as the methodology used to measure the effect of these novel techniques on user performance and workload.

Virtual reality-enhanced ultrasound guidance: a novel technique for intracardiac interventions.

Cardiopulmonary bypass surgery, although a highly invasive interventional approach leading to numerous complications, is still the most common therapy option for treating many forms of cardiac disease. We are currently engaged in a project designed to replace many bypass surgeries with less traumatic, minimally invasive intracardiac therapies. This project combines real-time intra-operative echocardiography with a virtual reality environment providing the surgeon with a broad range of valuable information. Pre-operative images, electrophysiological data, positions of magnetically tracked surgical instruments, and dynamic surgical target representations are among the data that can be presented to the surgeon to augment intra-operative ultrasound images. This augmented reality system is applicable to procedures such as mitral valve replacement and atrial septal defect repair, as well as ablation therapies for treatment of atrial fibrillation. Our goal is to develop a robust augmented reality system that will improve the efficacy of intracardiac treatments and broaden the range of cardiac surgeries that can be performed in a minimally invasive manner. This paper provides an overview of our interventional system and specific experiments that assess its pre-clinical performance.

2D ultrasound augmented by virtual tools for guidance of interventional procedures.

Many intracardiac procedures can currently be performed on the heart only after it has been arrested, and the patient has been placed on cardio-pulmonary bypass. We have developed a new method for operating on multiple targets inside the beating heart, and describe a procedure for accessing them under virtual-reality (VR)-assisted image guidance that combines real-time ultrasound with a virtual model of tools, and the surgical environment acquired from pre-operative images. This paper presents preliminary results aimed at assessing the operator's ability to accurately position and staple an artificial valve to a "valve orifice" within a cardiac phantom when guidance is performed via ultrasound alone, and with US augmented by the VR environment.

Image-guided laser projection for port placement in minimally invasive surgery.

We present an application of an augmented reality laser projection system in which procedure-specific optimal incision sites, computed from pre-operative image acquisition, are superimposed on a patient to guide port placement in minimally invasive surgery. Tests were conducted to evaluate the fidelity of computed and measured port configurations, and to validate the accuracy with which a surgical tool-tip can be placed at an identified virtual target. A high resolution volumetric image of a thorax phantom was acquired using helical computed tomography imaging. Oriented within the thorax, a phantom organ with marked targets was visualized in a virtual environment. A graphical interface enabled marking the locations of target anatomy, and calculation of a grid of potential port locations along the intercostal rib lines. Optimal configurations of port positions and tool orientations were determined by an objective measure reflecting image-based indices of surgical dexterity, hand-eye alignment, and collision detection. Intra-operative registration of the computed virtual model and the phantom anatomy was performed using an optical tracking system. Initial trials demonstrated that computed and projected port placement provided direct access to target anatomy with an accuracy of 2 mm.

Augmented reality laser projection device for surgery.

We have developed an augmented reality system capable of projecting preoperative plans directly onto a patient using rapidly scanned laser beams. Projected contours can typically represent cut paths, tumors or delineate boundaries of interest. The system can be used as part of, or a replacement for, conventional robotic Telesurgery systems. Because the graphics are projected, there is no degradation in surgeon's view due to optical components interposed between the surgeon's eye and the patient. This system has been designed to work with a common infrared 3D camera system used in image-guided surgery, and projects both visible and infrared beams. The IR beam enables surface digitization functions to be carried out using the camera. The clinical accuracy is in the range required by CAS procedures, around 1-2mm. The device will be particularly useful for executing precise preoperative plans and for teleconsultation applications, where planned or live consultations can be efficiently communicated to a less skilled local caregiver.

A navigation platform for guidance of beating heart transapical mitral valve repair.

Traditional surgical approaches for repairing diseased mitral valves (MVs) have relied on placing the patient on cardiopulmonary bypass (on pump), stopping the heart and accessing the arrested heart directly. However, because this approach has the potential for adverse neurological, vascular, and immunological sequelae, less invasive beating heart alternatives are desirable. Emerging beating heart techniques have been developed to offer high-risk patients MV repair using ultrasound guidance alone without stopping the heart. This paper describes the first porcine trials of the NeoChord DS1000 (Minnetonka, MN), employed to attach neochordae to a MV leaflet using the traditional ultrasound-guided protocol augmented by dynamic virtual geometric models. The distance errors of the tracked tool tip from the intended midline trajectory (5.2 ± 2.4 mm versus 16.8 ± 10.9 mm, p = 0.003), navigation times (16.7 ± 8.0 s versus 92.0 ± 84.5 s, p = 0.004), and total path lengths (225.2 ± 120.3 mm versus 1128.9 ± 931.1 mm, p = 0.003) were significantly shorter in the augmented ultrasound compared to navigation with ultrasound alone, indicating a substantial improvement in the safety and simplicity of the procedure.

Augmented reality image guidance improves navigation for beating heart mitral valve repair.

OBJECTIVE: Emerging off-pump beating heart valve repair techniques offer patients less invasive alternatives for mitral valve (MV) repair. However, most of these techniques rely on the limited spatial and temporal resolution of transesophageal echocardiography (TEE) alone, which can make tool visualization and guidance challenging. METHODS: Using a magnetic tracking system and integrated sensors, we created an augmented reality (AR) environment displaying virtual representations of important intracardiac landmarks registered to biplane TEE imaging. In a porcine model, we evaluated the AR guidance system versus TEE alone using the transapically delivered NeoChord DS1000 system to perform MV repair with chordal reconstruction. RESULTS: Successful tool navigation from left ventricular apex to MV leaflet was achieved in 12 of 12 and 9 of 12 (P = 0.2) attempts with AR imaging and TEE alone, respectively. The distance errors of the tracked tool tip from the intended midline trajectory (5.2 ± 2.4 mm vs 16.8 ± 10.9 mm, P = 0.003), navigation times (16.7 ± 8.0 seconds vs 92.0 ± 84.5 seconds, P = 0.004), and total path lengths (225.2 ± 120.3 mm vs 1128.9 ± 931.1 mm, P = 0.003) were significantly shorter in the AR-guided trials compared with navigation with TEE alone. Furthermore, the potential for injury to other intracardiac structures was nearly 40-fold lower when using the AR imaging for tool navigation. The AR guidance also seemed to shorten the learning curve for novice surgeons. CONCLUSIONS: Augmented reality-enhanced TEE facilitates more direct and safe intracardiac navigation of the NeoChord DS tool from left ventricular apex to MV leaflet. Tracked tool path results demonstrate fourfold improved accuracy, fivefold shorter navigation times, and overall improved safety with AR imaging guidance.

Investigating perioperative heart migration during robot-assisted coronary artery bypass grafting interventions.

OBJECTIVE: : For robot-assisted coronary artery bypass graft interventions, surgeons typically use a preoperative thoracic computed tomography scan of the patient to plan the procedure. However, the cardiac anatomy is prone to changes induced perioperatively in the effort to access the heart and surgical targets, which, in turn, may invalidate the initial plan. This article presents a method to estimate the perioperative heart migration, information which can be further exploited to refine the preoperative surgical plan. METHODS: : Tracked transesophageal ultrasound images of four patients' hearts were acquired at each stage in the procedure: before lung deflation, after lung deflation, and after both lung deflation and CO2 thoracic insufflation. Anatomic features of interest-the mitral and aortic valves-were identified from each dataset, and their movement between the different procedure stages was recorded and used to estimate the global heart displacement. Moreover, the local morphology of the features of interest was investigated to provide insight on the extent of the deformation the heart has undergone during the workflow. RESULTS: : The study suggested that the heart does undergo substantial displacement-on the order of 10 to 15 mm in each direction (axial, coronal, and sagittal) after lung deflation and CO2 thoracic insufflation. However, no significant differences (P > 0.1) were observed in the morphologic characteristics of the features of interest across the multiple workflow stages, suggesting that local deformations occur at a much smaller scale compared with the global migration. CONCLUSIONS: : The quantification of the perioperatively induced changes is critical to track the displacement of the heart and surgical targets. The recorded migration patterns should not be ignored but rather be used to update the surgical plan to better suit the intraoperative environment.

Evaluation of model-enhanced ultrasound-assisted interventional guidance in a cardiac phantom.

Minimizing invasiveness associated with cardiac procedures has led to limited visual access to the target tissues. To address these limitations, we have developed a visualization environment that integrates interventional ultrasound (US) imaging with preoperative anatomical models and virtual representations of the surgical instruments tracked in real time. In this paper, we present a comprehensive evaluation of our model-enhanced US-guidance environment by simulating clinically relevant interventions in vitro . We have demonstrated that model-enhanced US guidance provides a clinically desired targeting accuracy better than 3-mm rms and maintains this level of accuracy even in the case of image-to-patient misalignments that are often encountered in the clinic. These studies emphasize the benefits of integrating real-time imaging with preoperative data to enhance surgical navigation in the absence of direct vision during minimally invasive cardiac interventions.

Predicting target vessel location for improved planning of robot-assisted CABG procedures.

Prior to performing a robot-assisted coronary artery bypass grafting procedure, a pre-operative computed tomography scan is used to assess patient candidacy and to identify the location of the target vessel. The surgeon then determines the optimal port locations to ensure proper reach to the target with the robotic instruments, while assuming that the heart does not undergo any significant changes between the pre- and intra-operative stages. However, the peri-operative workflow itself leads to changes in heart position and consequently the intra-operative target vessel location. As such, the pre-operative plan must be adequately updated to adjust the target vessel location to better suit the intraoperative condition. Here we propose a technique to predict the position of the peri-operative target vessel location with approximately 3.5 mm RMS accuracy. We believe this technique will potentially reduce the rate of conversion of robot-assisted procedures to traditional open-chest surgery due to poor planning.

Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study.

The shift to minimally invasive abdominal surgery has increased reliance on image guidance during surgical procedures. However, these images are most often presented independently, increasing the cognitive workload for the surgeon and potentially increasing procedure time. When warm ischemia of an organ is involved, time is an important factor to consider. To address these limitations, we present a more intuitive visualization that combines images in a common augmented reality environment. In this paper, we assess surgeon performance under the guidance of the conventional visualization system and our fusion system using a phantom study that mimics the tumour resection of partial nephrectomy. The RMS error between the fused images was 2.43mm, which is sufficient for our purposes. A faster planning time for the resection was achieved using our fusion visualization system. This result is a positive step towards decreasing risks associated with long procedure times in minimally invasive abdominal interventions.

Inside the beating heart: an in vivo feasibility study on fusing pre- and intra-operative imaging for minimally invasive therapy.

OBJECTIVE: An interventional system for minimally invasive cardiac surgery was developed for therapy delivery inside the beating heart, in absence of direct vision. METHOD: A system was developed to provide a virtual reality (VR) environment that integrates pre-operative imaging, real-time intra-operative guidance using 2D trans-esophageal ultrasound, and models of the surgical tools tracked using a magnetic tracking system. Detailed 3D dynamic cardiac models were synthesized from high-resolution pre-operative MR data and registered within the intra-operative imaging environment. The feature-based registration technique was employed to fuse pre- and intra-operative data during in vivo intracardiac procedures on porcine subjects. RESULTS: This method was found to be suitable for in vivo applications as it relies on easily identifiable landmarks, and hence, it ensures satisfactory alignment of pre- and intra-operative anatomy in the region of interest (4.8 mm RMS alignment accuracy) within the VR environment. Our initial experience in translating this work to guide intracardiac interventions, such as mitral valve implantation and atrial septal defect repair demonstrated feasibility of the methods. CONCLUSION: Surgical guidance in the absence of direct vision and with no exposure to ionizing radiation was achieved, so our virtual environment constitutes a feasible candidate for performing various off-pump intracardiac interventions.

Targeting accuracy under model-to-subject misalignments in model-guided cardiac surgery.

In image-guided interventions, anatomical models of organs are often generated from pre-operative images and further employed in planning and guiding therapeutic procedures. However, the accuracy of these models, along with their registration to the subject are crucial for successful therapy delivery. These factors are amplified when manipulating soft tissue undergoing large deformations, such as the heart. When used in guiding beating-heart procedures, pre-operative models may not be sufficient for guidance and they are often complemented with real-time, intra-operative cardiac imaging. Here we demonstrate via in vitro endocardial "therapy" that ultrasound-enhanced model-guided navigation provides sufficient guidance to preserve a clinically-desired targeting accuracy of under 3 mm independently of the model-to-subject misregistrations. These results emphasize the direct benefit of integrating real-time imaging within intra-operative visualization environments considering that model-to-subject misalignments are often encountered clinically.

Image guidance for spinal facet injections using tracked ultrasound.

Anesthetic nerve blocks are a common therapy performed in hospitals around the world to alleviate acute and chronic pain. Tracking systems have shown considerable promise in other forms of therapy, but little has been done to apply this technology in the field of anesthesia. We are developing a guidance system for combining tracked needles with non-invasive ultrasound (US) and patient-specific geometric models. In experiments with phantoms two augmented reality (AR) guidance systems were compared to the exclusive use of US for lumbar facet injection therapy. Anesthetists and anesthesia residents were able to place needles within 0.57 mm of the intended targets using our AR systems compared to 5.77 mm using US alone. A preliminary cadaver study demonstrated the system was able to accurately place radio opaque dye on targets. The combination of real time US with tracked tools and AR guidance has the potential to replace CT and fluoroscopic guidance, thus reducing radiation dose to patients and clinicians, as well as reducing health care costs.

Off-pump atrial septal defect closure using the universal cardiac introducer®: creation of models of atrial septal defects in the pig access and surgical technique.

OBJECTIVE: : Optimal atrial septal defect (ASD) closure should combine off-pump techniques with the effectiveness and versatility of open-heart techniques. We report our experience with off-pump ASD closure using the Universal Cardiac Introducer (UCI) in a porcine model. The goal was to create an ASD over the fossa ovale (FO) and position a patch over the ASD under ultrasound (US) imaging and augmented virtual reality guidance. METHODS: : An US probe (tracked with a magnetic tracking system) was positioned into the esophagus (transesophageal echocardiographic probe) for real-time image-guidance. The right atrium (RA) of six pigs was exposed via a right lateral thoracotomy or medial sternotomy. The UCI was attached to the RA wall. A punching tool was introduced via the UCI, navigated and positioned, under US guidance, to create an ASD into the FO. A patch with its holder and a stapling device were introduced into the RA via the UCI. The patch was positioned on the ASD. Occlusion of the ASD was determined using US and Doppler imaging. RESULTS: : The FO membrane was excised successfully in all animals. US image-guidance provided excellent visualization. The patch was positioned in all cases with complete occlusion of the ASD. The stapling device proved too bulky, impeding circumferential positioning. CONCLUSIONS: : Using the UCI, ASD closure was safe and feasible. US imaging, combined with virtual and augmented reality provided accurate navigating and positioning. This study also provided valuable information on the future design of anchoring devices for intracardiac procedures.

Dynamic cardiac mapping on patient-specific cardiac models.

Minimally invasive techniques for electrophysiological cardiac data mapping and catheter ablation therapy have been driven through advancements in computer-aided technologies, including magnetic tracking systems, and virtual and augmented-reality environments. The objective of this work is to extend current cardiac mapping techniques to collect and display data in the temporal domain, while mapping on patient-specific cardiac models. This paper details novel approaches to collecting spatially tracked cardiac electrograms, registering the data with a patient-specific cardiac model, and interpreting the data directly on the model surface, with the goal of giving a more comprehensive cardiac mapping system in comparison to current systems. To validate the system, laboratory studies were conducted to assess the accuracy of navigating to both physical and virtual landmarks. Subsequent to the laboratory studies, an in-vivo porcine experiment was conducted to assess the systems overall ability to collect spatial tracked electrophysiological data, and map directly onto a cardiac model. The results from these experiments show the new dynamic cardiac mapping system was able to maintain high accuracy of locating physical and virtual landmarks, while creating a dynamic cardiac map displayed on a dynamic cardiac surface model.

Surgical accuracy under virtual reality-enhanced ultrasound guidance: an in vitro epicardial dynamic study.

In the context of our ongoing objective to reduce morbidity associated with cardiac interventions, minimizing invasiveness has inevitably led to more limited visual access to the target tissues. To ameliorate these challenges, we provide the surgeons with a complex visualization environment that integrates interventional ultrasound imaging augmented with pre-operative anatomical models and virtual surgical instruments within a virtual reality environment. In this paper we present an in vitro study on a cardiac phantom aimed at assessing the feasibility and targeting accuracy of our surgical system in comparison to traditional ultrasound imaging for intra-operative surgical guidance. The 'therapy delivery' was modeled in the context of a blinded procedure, mimicking a closed-chest intervention. Four users navigated a tracked pointer to a target, under guidance provide by either US imaging or virtual reality-enhanced ultrasound. A 2.8 mm RMS targeting error was achieved using our novel surgical system, which is adequate from both a clinical and engineering perspective, under the inherent procedure requirements and limitations of the system.

Virtual reality-enhanced ultrasound guidance for atrial ablation: in vitro epicardial study.

In an effort to reduce morbidity of cardiac interventions, minimizing invasiveness inevitably leads to limited visual access to the surgical targets. To address these limitations, we provide the surgeons with a robust visualization environment that integrates interventional ultrasound imaging augmented with pre-operative anatomical models and virtual surgical instruments within a virtual reality environment. Here we present an in vitro study on a cardiac phantom that mimics an ablation therapy procedure, which allows us to assess the feasibility of our surgical system in comparison to traditional intra-operative ultrasound imaging. Following surgical target identification via an electro-anatomical model, the "ablation procedure" is performed blindly. A 2.8 mm RMS targeting error is achieved using our novel surgical system. This level of accuracy is adequate from both a clinical and engineering perspective, under the inherent procedure requirements and limitations of the system.