Feasibility and limitations of an automated 2D-3D rigid image registration system for complex endovascular aortic procedures.

PURPOSE: To examine the feasibility of an automated 2-dimensional (2D) to 3- dimensional (3D) image registration system to simplify the navigational challenges faced in complex endovascular aortic procedures. METHODS: An automated 2D-3D image registration system was used to overlay pre-acquired 3D computed tomography images onto fluoroscopy images taken during endovascular aneurysm repair. Errors between the 3D overlay and digital subtraction angiograms were measured and correlated with aortic neck angulation. A mean discrepancy < or =3 mm was considered clinically acceptable. RESULTS: There was a strong correlation between maximum neck angulation and maximum registration error (Pearson's r = 0.75). Aortas with a maximum neck angulation < or =30 degrees had a mean error of 2.5+/-1.2 mm, whereas aortas with neck angulation >30 degrees had a mean error of 6.2+/-2.5 mm (p<0.0001). CONCLUSION: The major source of registration errors is aortic deformation caused by the presence of the introducer and endovascular graft. Further work is required if this technology is to be routinely applied to severely angulated aortic anatomy.

Echocardiography to magnetic resonance image registration for use in image-guided cardiac catheterization procedures.

We present a robust method to register three-dimensional echocardiography (echo) images to magnetic resonance images (MRI) based on anatomical features, which is designed to be used in the registration pipeline for overlaying MRI-derived roadmaps onto two-dimensional live x-ray images during cardiac catheterization procedures. The features used in image registration are the endocardial surface of the left ventricle and the centre line of the descending aorta. The MR-derived left ventricle surface is generated using a fully automated algorithm, and the echo-derived left ventricle surface is produced using a semi-automatic segmentation method provided by the QLab software (Philips Healthcare) that it is routinely used in clinical practice. We test our method on data from six volunteers and four patients. We validated registration accuracy using two methods: the first calculated a root mean square distance error using expert identified anatomical landmarks, and the second method used catheters as landmarks in two clinical electrophysiology procedures. Results show a mean error of 4.1 mm, which is acceptable for our clinical application, and no failed registrations were observed. In addition, our algorithm works on clinical data, is fast and only requires a small amount of manual input, and so it is applicable for use during cardiac catheterization procedures.

Postoperative calculation of acetabular cup position using 2-D-3-D registration.

A method to accurately measure the position and orientation of an acetabular cup implant from postoperative X-rays has been designed and validated. The method uses 2-D-3-D registration to align both the prosthesis and the preoperative computed tomography (CT) volume to the X-ray image. This allows the position of the implant to be calculated with respect to a CT-based surgical plan. Experiments have been carried out using ten sets of patient data. A conventional plain-film measurement technique was also investigated. A gold standard implant position and orientation was calculated using postoperative CT. Results show our method to be significantly more accurate than the plain-film method for calculating cup anteversion. Cup orientation and position could be measured to within a mean absolute error of 1.4 mm or degrees.

Registration of Multiview Echocardiography Sequences Using a Subspace Error Metric.

OBJECTIVE: 3-D +t echocardiography (3DtE) is widely employed for the assessment of left ventricular anatomy and function. However, the information derived from 3DtE images can be affected by the poor image quality and the limited field of view. Registration of multiview 3DtE sequences has been proposed to compound images from different acoustic windows, therefore improving both image quality and coverage. We propose a novel subspace error metric for an automatic and robust registration of multiview intrasubject 3DtE sequences. METHODS: The proposed metric employs linear dimensionality reduction to exploit the similarity in the temporal variation of multiview 3DtE sequences. The use of a low-dimensional subspace for the computation of the error metric reduces the influence of image artefacts and noise on the registration optimization, resulting in fast and robust registrations that do not require a starting estimate. RESULTS: The accuracy, robustness, and execution time of the proposed registration were thoroughly validated. Results on 48 pairwise multiview 3DtE registrations show the proposed error metric to outperform a state-of-the-art phase-based error metric, with improvements in median/75th percentile of the target registration error of 21%/31% and an improvement in mean execution time of 45%. CONCLUSION: The proposed subspace error metric outperforms sum-of-squared differences and phase-based error metrics for the registration of multiview 3DtE sequences in terms of accuracy, robustness, and execution time. SIGNIFICANCE: The use of the proposed subspace error metric has the potential to replace standard image error metrics for a robust and automatic registration of multiview 3DtE sequences.

Self-calibrating 3D-ultrasound-based bone registration for minimally invasive orthopedic surgery.

Intraoperative freehand three-dimensional (3-D) ultrasound (3D-US) has been proposed as a noninvasive method for registering bones to a preoperative computed tomography image or computer-generated bone model during computer-aided orthopedic surgery (CAOS). In this technique, an US probe is tracked by a 3-D position sensor and acts as a percutaneous device for localizing the bone surface. However, variations in the acoustic properties of soft tissue, such as the average speed of sound, can introduce significant errors in the bone depth estimated from US images, which limits registration accuracy. We describe a new self-calibrating approach to US-based bone registration that addresses this problem, and demonstrate its application within a standard registration scheme. Using realistic US image data acquired from 6 femurs and 3 pelves of intact human cadavers, and accurate Gold Standard registration transformations calculated using bone-implanted fiducial markers, we show that self-calibrating registration is significantly more accurate than a standard method, yielding an average root mean squared target registration error of 1.6 mm. We conclude that self-calibrating registration results in significant improvements in registration accuracy for CAOS applications over conventional approaches where calibration parameters of the 3D-US system remain fixed to values determined using a preoperative phantom-based calibration.

Regional Differences in End-Diastolic Volumes between 3D Echo and CMR in HLHS Patients.

Ultrasound is commonly thought to underestimate ventricular volumes compared to magnetic resonance imaging (MRI), although the reason for this and the spatial distribution of the volume difference is not well understood. In this paper, we use landmark-based image registration to spatially align MRI and ultrasound images from patients with hypoplastic left heart syndrome and carry out a qualitative and quantitative spatial comparison of manual segmentations of the ventricular volume obtained from the respective modalities. In our experiments, we have found a trend showing volumes estimated from ultrasound to be smaller than those obtained from MRI (by approximately up to 20 ml), and that important contributors to this difference are the presence of artifacts such as shadows in the echo images and the different criteria to include or exclude image features as part of the ventricular volume.

Alignment of sparse freehand 3-D ultrasound with preoperative images of the liver using models of respiratory motion and deformation.

We present a method for alignment of an interventional plan to optically tracked two-dimensional intraoperative ultrasound (US) images of the liver. Our clinical motivation is to enable the accurate transfer of information from three-dimensional preoperative imaging modalities [magnetic resonance (MR) or computed tomography (CT)] to intraoperative US to aid needle placement for thermal ablation of liver metastases. An initial rigid registration to intraoperative coordinates is obtained using a set of US images acquired at maximum exhalation. A preprocessing step is applied to both the preoperative images and the US images to produce evidence of corresponding structures. This yields two sets of images representing classification of regions as vessels. The registration then proceeds using these images. The preoperative images and plan are then warped to correspond to a single US slice acquired at an unknown point in the breathing cycle where the liver is likely to have moved and deformed relative to the preoperative image. Alignment is constrained using a patient-specific model of breathing motion and deformation. Target registration error is estimated by carrying out simulation experiments using resliced MR volumes to simulate real US and comparing the registration results to a "bronze-standard" registration performed on the full MR volume. Finally, the system is tested using real US and verified using visual inspection.

Standardized evaluation methodology for 2-D-3-D registration.

In the past few years, a number of two-dimensional (2-D) to three-dimensional (3-D) (2-D-3-D) registration algorithms have been introduced. However, these methods have been developed and evaluated for specific applications, and have not been directly compared. Understanding and evaluating their performance is therefore an open and important issue. To address this challenge we introduce a standardized evaluation methodology, which can be used for all types of 2-D-3-D registration methods and for different applications and anatomies. Our evaluation methodology uses the calibrated geometry of a 3-D rotational X-ray (3DRX) imaging system (Philips Medical Systems, Best, The Netherlands) in combination with image-based 3-D-3-D registration for attaining a highly accurate gold standard for 2-D X-ray to 3-D MR/CT/3DRX registration. Furthermore, we propose standardized starting positions and failure criteria to allow future researchers to directly compare their methods. As an illustration, the proposed methodology has been used to evaluate the performance of two 2-D-3-D registration techniques, viz. a gradient-based and an intensity-based method, for images of the spine. The data and gold standard transformations are available on the internet (http://www.isi.uu.nl/Research/Databases/).

4D Blood Flow Reconstruction Over the Entire Ventricle From Wall Motion and Blood Velocity Derived From Ultrasound Data.

We demonstrate a new method to recover 4D blood flow over the entire ventricle from partial blood velocity measurements using multiple 3D+t colour Doppler images and ventricular wall motion estimated using 3D+t BMode images. We apply our approach to realistic simulated data to ascertain the ability of the method to deal with incomplete data, as typically happens in clinical practice. Experiments using synthetic data show that the use of wall motion improves velocity reconstruction, shows more accurate flow patterns and improves mean accuracy particularly when coverage of the ventricle is poor. The method was applied to patient data from 6 congenital cases, producing results consistent with the simulations. The use of wall motion produced more plausible flow patterns and reduced the reconstruction error in all patients.

Intensity-based 2-D-3-D registration of cerebral angiograms.

We propose a new method for aligning three-dimensional (3-D) magnetic resonance angiography (MRA) with 2-D X-ray digital subtraction angiograms (DSA). Our method is developed from our algorithm to register computed tomography volumes to X-ray images based on intensity matching of digitally reconstructed radiographs (DRRs). To make the DSA and DRR more similar, we transform the MRA images to images of the vasculature and set to zero the contralateral side of the MRA to that imaged with DSA. We initialize the search for a match on a user defined circular region of interest. We have tested six similarity measures using both unsegmented MRA and three segmentation variants of the MRA. Registrations were carried out on images of a physical neuro-vascular phantom and images obtained during four neuro-vascular interventions. The most accurate and robust registrations were obtained using the pattern intensity, gradient difference, and gradient correlation similarity measures, when used in conjunction with the most sophisticated MRA segmentations. Using these measures, 95% of the phantom start positions and 82% of the clinical start positions were successfully registered. The lowest root mean square reprojection errors were 1.3 mm (standard deviation 0.6) for the phantom and 1.5 mm (standard deviation 0.9) for the clinical data sets. Finally, we present a novel method for the comparison of similarity measure performance using a technique borrowed from receiver operator characteristic analysis.

Personalising population-based respiratory motion models of the heart using neighbourhood approximation based on learnt anatomical features.

Respiratory motion models have been proposed for the estimation and compensation of respiratory motion during image acquisition and image-guided interventions on organs in the chest and abdomen. However, such techniques are not commonly used in the clinic. Subject-specific motion models require a dynamic calibration scan that interrupts the clinical workflow and is often impractical to acquire, while population-based motion models are not as accurate as subject-specific motion models. To address this lack of accuracy, we propose a novel personalisation framework for population-based respiratory motion models and demonstrate its application to respiratory motion of the heart. The proposed method selects a subset of the population sample which is more likely to represent the cardiac respiratory motion of an unseen subject, thus providing a more accurate motion model. The selection is based only on anatomical features of the heart extracted from a static image. The features used are learnt using a neighbourhood approximation technique from a set of training datasets for which respiratory motion estimates are available. Results on a population sample of 28 adult healthy volunteers show average improvements in estimation accuracy of 20% compared to a standard population-based motion model, with an average value for the 50th and 95th quantiles of the estimation error of 1.6mm and 4.7 mm respectively. Furthermore, the anatomical features of the heart most strongly correlated to respiratory motion are investigated for the first time, showing the features on the apex in proximity to the diaphragm and the rib cage, on the left ventricle and interventricular septum to be good predictors of the similarity in cardiac respiratory motion.

Multiview diffeomorphic registration: application to motion and strain estimation from 3D echocardiography.

This paper presents a new registration framework for quantifying myocardial motion and strain from the combination of multiple 3D ultrasound (US) sequences. The originality of our approach lies in the estimation of the transformation directly from the input multiple views rather than from a single view or a reconstructed compounded sequence. This allows us to exploit all spatiotemporal information available in the input views avoiding occlusions and image fusion errors that could lead to some inconsistencies in the motion quantification result. We propose a multiview diffeomorphic registration strategy that enforces smoothness and consistency in the spatiotemporal domain by modeling the 4D velocity field continuously in space and time. This 4D continuous representation considers 3D US sequences as a whole, therefore allowing to robustly cope with variations in heart rate resulting in different number of images acquired per cardiac cycle for different views. This contributes to the robustness gained by solving for a single transformation from all input sequences. The similarity metric takes into account the physics of US images and uses a weighting scheme to balance the contribution of the different views. It includes a comparison both between consecutive images and between a reference and each of the following images. The strain tensor is computed locally using the spatial derivatives of the reconstructed displacement fields. Registration and strain accuracy were evaluated on synthetic 3D US sequences with known ground truth. Experiments were also conducted on multiview 3D datasets of 8 volunteers and 1 patient treated by cardiac resynchronization therapy. Strain curves obtained from our multiview approach were compared to the single-view case, as well as with other multiview approaches. For healthy cases, the inclusion of several views improved the consistency of the strain curves and reduced the number of segments where a non-physiological strain pattern was observed. For the patient, the improvement (pacing ON vs. OFF) in synchrony of regional strain correlated with clinician blind assessment and could be seen more clearly when using the multiview approach.

A novel Bayesian respiratory motion model to estimate and resolve uncertainty in image-guided cardiac interventions.

In image-guided cardiac interventions, respiratory motion causes misalignments between the pre-procedure roadmap of the heart used for guidance and the intra-procedure position of the heart, reducing the accuracy of the guidance information and leading to potentially dangerous consequences. We propose a novel technique for motion-correcting the pre-procedural information that combines a probabilistic MRI-derived affine motion model with intra-procedure real-time 3D echocardiography (echo) images in a Bayesian framework. The probabilistic model incorporates a measure of confidence in its motion estimates which enables resolution of the potentially conflicting information supplied by the model and the echo data. Unlike models proposed so far, our method allows the final motion estimate to deviate from the model-produced estimate according to the information provided by the echo images, so adapting to the complex variability of respiratory motion. The proposed method is evaluated using gold-standard MRI-derived motion fields and simulated 3D echo data for nine volunteers and real 3D live echo images for four volunteers. The Bayesian method is compared to 5 other motion estimation techniques and results show mean/max improvements in estimation accuracy of 10.6%/18.9% for simulated echo images and 20.8%/41.5% for real 3D live echo data, over the best comparative estimation method.

Multi-view 3D echocardiography compounding based on feature consistency.

Echocardiography (echo) is a widely available method to obtain images of the heart; however, echo can suffer due to the presence of artefacts, high noise and a restricted field of view. One method to overcome these limitations is to use multiple images, using the 'best' parts from each image to produce a higher quality 'compounded' image. This paper describes our compounding algorithm which specifically aims to reduce the effect of echo artefacts as well as improving the signal-to-noise ratio, contrast and extending the field of view. Our method weights image information based on a local feature coherence/consistency between all the overlapping images. Validation has been carried out using phantom, volunteer and patient datasets consisting of up to ten multi-view 3D images. Multiple sets of phantom images were acquired, some directly from the phantom surface, and others by imaging through hard and soft tissue mimicking material to degrade the image quality. Our compounding method is compared to the original, uncompounded echocardiography images, and to two basic statistical compounding methods (mean and maximum). Results show that our method is able to take a set of ten images, degraded by soft and hard tissue artefacts, and produce a compounded image of equivalent quality to images acquired directly from the phantom. Our method on phantom, volunteer and patient data achieves almost the same signal-to-noise improvement as the mean method, while simultaneously almost achieving the same contrast improvement as the maximum method. We show a statistically significant improvement in image quality by using an increased number of images (ten compared to five), and visual inspection studies by three clinicians showed very strong preference for our compounded volumes in terms of overall high image quality, large field of view, high endocardial border definition and low cavity noise.

Hybrid echo and x-ray image guidance for cardiac catheterization procedures by using a robotic arm: a feasibility study.

We present a feasibility study on hybrid echocardiography (echo) and x-ray image guidance for cardiac catheterization procedures. A self-tracked, remotely operated robotic arm with haptic feedback was developed that attached to a standard x-ray table. This was used to safely manipulate a three-dimensional (3D) trans-thoracic echo probe during simultaneous x-ray fluoroscopy and echo acquisitions. By a combination of calibration and tracking of the echo and x-ray systems, it was possible to register the 3D echo images with the 2D x-ray images. Visualization of the combined data was achieved by either overlaying triangulated surfaces extracted from segmented echo data onto the x-ray images or by overlaying volume rendered 3D echo data. Furthermore, in order to overcome the limited field of view of the echo probe, it was possible to create extended field of view (EFOV) 3D echo images by co-registering multiple tracked echo data to generate larger roadmaps for procedure guidance. The registration method was validated using a cross-wire phantom and showed a 2D target registration error of 3.5 mm. The clinical feasibility of the method was demonstrated during two clinical cases for patients undergoing cardiac pacing studies. The EFOV technique was demonstrated using two healthy volunteers.

Non-rigid reconstruction of the beating heart surface for minimally invasive cardiac surgery.

This paper presents a new method to reconstruct the beating heart surface based on the non-rigid structure from motion technique using preprocessed endoscopic images. First the images captured at the same phase within each heart cycle are automatically extracted from the original image sequence to reduce the dimension of the deformation subspace. Then the remaining residual non-rigid motion is restricted to lie within a low-dimensional subspace and a probabilistic model is used to recover the 3D structure and camera motion simultaneously. Outliers are removed iteratively based on the reprojection error, Missing data are also recovered with an Expectation Maximization algorithm. As a result the camera can move around the operation scene to build a 3D surface with a wide field-of-view for intra-operative procedures. The method has been evaluated with synthetic data, heart phantom data, and in vivo data from a da Vinci surgical system.

Image-to-physical registration for image-guided interventions using 3-D ultrasound and an ultrasound imaging model.

We present a technique for automatic intensity-based image-to-physical registration of a 3-D segmentation for image-guided interventions. The registration aligns the segmentation with tracked and calibrated 3-D ultrasound (US) images of the target region. The technique uses a probabilistic framework and explicitly incorporates a model of the US image acquisition process. The rigid body registration parameters are varied to maximise the likelihood that the real US image(s) were formed using the US imaging model from the probe transducer position. The proposed technique is validated on images segmented from cardiac magnetic resonance imaging (MRI) data and 3-D US images acquired from 3 volunteers and 1 patient. We show that the accuracy of the algorithm is 2.6-4.2mm and the capture range is 9-18mm. The proposed technique has the potential to provide accurate image-to-physical registrations for a range of image guidance applications.

Evaluation of a robotic arm for echocardiography to X-ray image registration during cardiac catheterization procedures.

We present an initial evaluation of a robotic arm for positioning a 3D echo probe during cardiac catheterization procedures. By tracking the robotic arm, X-ray table and X-ray C-arm, we are able to register the 3D echo images with live 2D X-ray images. In addition, we can also use tracking data from the robotic arm combined with system calibrations to create extended field of view 3D echo images. Both these features can be used for roadmapping to guide cardiac catheterization procedures. We have carried out a validation experiment of our registration method using a cross-wire phantom. Results show our method to be accurate to 3.5 mm. We have successfully demonstrated the creation of the extended field of view data on 2 healthy volunteers and the registration of echo and X-ray data on 1 patient undergoing a pacing study.

Instantiation and registration of statistical shape models of the femur and pelvis using 3D ultrasound imaging.

Statistical shape modelling potentially provides a powerful tool for generating patient-specific, 3D representations of bony anatomy for computer-aided orthopaedic surgery (CAOS) without the need for a preoperative CT scan. Furthermore, freehand 3D ultrasound (US) provides a non-invasive method for digitising bone surfaces in the operating theatre that enables a much greater region to be sampled compared with conventional direct-contact (i.e., pointer-based) digitisation techniques. In this paper, we describe how these approaches can be combined to simultaneously generate and register a patient-specific model of the femur and pelvis to the patient during surgery. In our implementation, a statistical deformation model (SDM) was constructed for the femur and pelvis by performing a principal component analysis on the B-spline control points that parameterise the freeform deformations required to non-rigidly register a training set of CT scans to a carefully segmented template CT scan. The segmented template bone surface, represented by a triangulated surface mesh, is instantiated and registered to a cloud of US-derived surface points using an iterative scheme in which the weights corresponding to the first five principal modes of variation of the SDM are optimised in addition to the rigid-body parameters. The accuracy of the method was evaluated using clinically realistic data obtained on three intact human cadavers (three whole pelves and six femurs). For each bone, a high-resolution CT scan and rigid-body registration transformation, calculated using bone-implanted fiducial markers, served as the gold standard bone geometry and registration transformation, respectively. After aligning the final instantiated model and CT-derived surfaces using the iterative closest point (ICP) algorithm, the average root-mean-square distance between the surfaces was 3.5mm over the whole bone and 3.7mm in the region of surgical interest. The corresponding distances after aligning the surfaces using the marker-based registration transformation were 4.6 and 4.5mm, respectively. We conclude that despite limitations on the regions of bone accessible using US imaging, this technique has potential as a cost-effective and non-invasive method to enable surgical navigation during CAOS procedures, without the additional radiation dose associated with performing a preoperative CT scan or intraoperative fluoroscopic imaging. However, further development is required to investigate errors using error measures relevant to specific surgical procedures.

Multispectral MR to X-ray registration of vertebral bodies by generating CT-like data.

A new method for MR to X-ray registration is presented. Based on training data, consisting of registered multispectral MR and CT data, a function is defined that maps multispectral MR data to CT-like data. For new subjects for which multispectral MR data have been acquired, the mapping function is used to generate a corresponding CT-like dataset. The CT-like image is subsequently used for registration to X-ray data, using gradient-based registration. Preliminary experiments indicate that MR to X-ray registration using this method is more accurate and has a larger capture range than gradient-based registration applied directly to MR data.