Extraction of airways from CT (EXACT'09).

This paper describes a framework for establishing a reference airway tree segmentation, which was used to quantitatively evaluate fifteen different airway tree extraction algorithms in a standardized manner. Because of the sheer difficulty involved in manually constructing a complete reference standard from scratch, we propose to construct the reference using results from all algorithms that are to be evaluated. We start by subdividing each segmented airway tree into its individual branch segments. Each branch segment is then visually scored by trained observers to determine whether or not it is a correctly segmented part of the airway tree. Finally, the reference airway trees are constructed by taking the union of all correctly extracted branch segments. Fifteen airway tree extraction algorithms from different research groups are evaluated on a diverse set of twenty chest computed tomography (CT) scans of subjects ranging from healthy volunteers to patients with severe pathologies, scanned at different sites, with different CT scanner brands, models, and scanning protocols. Three performance measures covering different aspects of segmentation quality were computed for all participating algorithms. Results from the evaluation showed that no single algorithm could extract more than an average of 74% of the total length of all branches in the reference standard, indicating substantial differences between the algorithms. A fusion scheme that obtained superior results is presented, demonstrating that there is complementary information provided by the different algorithms and there is still room for further improvements in airway segmentation algorithms.

Mediastinal atlas creation from 3-D chest computed tomography images: application to automated detection and station mapping of lymph nodes.

One important aspect of lung cancer staging is the assessment of mediastinal lymph nodes in 3-D chest computed tomography (CT) images. In the current clinical routine this is done manually by analyzing the 3-D CT image slice by slice to find nodes, evaluate them quantitatively, and assign labels to them for describing the clinical and pathologic extent of metastases. In this paper we present a method to automate the process of lymph node detection and labeling by creation of a mediastinal average image and a novel lymph node atlas containing probability maps for mediastinal, aortic, and N1 nodes. Utilizing a fast deformable registration approach to match the atlas with CT images of new patients, our method can maintain an acceptable runtime. In comparison to previously published methods for mediastinal lymph node detection and labeling it also shows a good sensitivity and positive predictive value.

Development and comparison of new hybrid motion tracking for bronchoscopic navigation.

This paper presents a new hybrid camera motion tracking method for bronchoscopic navigation combining SIFT, epipolar geometry analysis, Kalman filtering, and image registration. In a thorough evaluation, we compare it to state-of-the-art tracking methods. Our hybrid algorithm for predicting bronchoscope motion uses SIFT features and epipolar constraints to obtain an estimate for inter-frame pose displacements and Kalman filtering to find an estimate for the magnitude of the motion. We then execute bronchoscope tracking by performing image registration initialized by these estimates. This procedure registers the actual bronchoscopic video and the virtual camera images generated from 3D chest CT data taken prior to bronchoscopic examination for continuous bronchoscopic navigation. A comparative assessment of our new method and the state-of-the-art methods is performed on actual patient data and phantom data. Experimental results from both datasets demonstrate a significant performance boost of navigation using our new method. Our hybrid method is a promising means for bronchoscope tracking, and outperforms other methods based solely on Kalman filtering or image features and image registration.

Real-time marker-free patient registration for electromagnetic navigated bronchoscopy: a phantom study.

PURPOSE: To perform bronchoscopy safely and smoothly, it is very important to develop a bronchoscopic guidance system. Transbronchial lung biopsy (TBLB) with a bronchoscopic guidance system especially should permit safe image-guided procedures. Recently, electromagnetic tracking (EMT) is utilized to track the tip of the bronchoscope camera in real time. For most tracking methods using position sensors, registration between tracking data and previously acquired reference image data, such as CT image, is performed using natural landmarks of the patient or fiducial markers attached to the patient, whose positions need to be measured manually by the physician before the actual bronchoscopy. Therefore, this paper proposes a marker-free CT-to-patient registration method utilizing bronchoscope's position and orientation obtained by the EMT. METHODS: We developed a guidance system that is able to track the tip of the bronchoscope camera in real time. In the case of a guidance system that uses position sensors, natural landmarks of the patient or fiducial markers attached to the patient are needed to obtain the correspondence between EMT outputs and previously acquired reference image data, such as CT image. This paper proposes a registration method without landmarks or fiducials by estimating the transformation matrix between the patient and the CT image taken prior to the bronchoscopic examination. This estimation is performed by computing correspondences between the outputs of the EMT sensor and airways extracted from the CT image. As ambiguities between EMT measurements and their corresponding airway branches may arise at airway bifurcations, we introduce a stable airway branch selection mechanism for improving the robustness of the estimation of the transformation matrix. To evaluate the performance of the proposed method, we applied the method to a rubber bronchial phantom and added virtual breathing motion to the sensor output. RESULTS: Experimental results show that the accuracy of our proposed method is within 2.0-3.0 mm (without breathing motion) and 2.5-3.5 mm (with breathing motion). The proposed method could also track a bronchoscope camera in real time. CONCLUSIONS: We developed a method for CT-to-patient registration using a position sensor without fiducial markers and natural landmarks. Endoscopic guided biopsy of lung lesions is feasible using a marker-free CT-to-patient registration method.

Robust bronchoscope motion tracking using sequential Monte Carlo methods in navigated bronchoscopy: dynamic phantom and patient validation.

PURPOSE: Accurate and robust estimates of camera position and orientation in a bronchoscope are required for navigation. Fusion of pre-interventional information (e.g., CT, MRI, or US) and intra-interventional information (e.g., bronchoscopic video) were incorporated into a navigation system to provide physicians with an augmented reality environment for bronchoscopic interventions. METHODS: Two approaches were used to predict bronchoscope movements by incorporating sequential Monte Carlo (SMC) simulation including (1) image-based tracking techniques and (2) electromagnetic tracking (EMT) methods. SMC simulation was introduced to model ambiguities or uncertainties that occurred in image- and EMT-based bronchoscope tracking. Scale invariant feature transform (SIFT) features were employed to overcome the limitations of image-based motion tracking methods. Validation was performed on five phantom and ten human case datasets acquired in the supine position. RESULTS: For dynamic phantom validation, the EMT-SMC simulation method improved the tracking performance of the successfully registered bronchoscopic video frames by 12.7% compared with a hybrid-based method. In comparisons between tracking results and ground truth, the accuracy of the EMT-SMC simulation method was 1.51 mm (positional error) and 5.44° (orientation error). During patient assessment, the SIFT-SMC simulation scheme was more stable or robust than a previous image-based approach for bronchoscope motion estimation, showing 23.6% improvement of successfully tracked frames. Comparing the estimates of our method to ground truth, the position and orientation errors are 3.72 mm and 10.2°, while those of our previous image-based method were at least 7.77 mm and 19.3°. The computational times of our EMT- and SIFT-SMC simulation methods were 0.9 and 1.2 s per frame, respectively. CONCLUSION: The SMC simulation method was developed to model ambiguities that occur in bronchoscope tracking. This method more stably and accurately predicts the bronchoscope camera position and orientation parameters, reducing uncertainties due to problematic bronchoscopic video frames and airway deformation during intra-bronchoscopy navigation.

Reconstruction of 3-D histology images by simultaneous deformable registration.

The reconstruction of histology sections into a 3-D volume receives increased attention due to its various applications in modern medical image analysis. To guarantee a geometrically coherent reconstruction, we propose a new way to register histological sections simuItaneously to previously acquired reference images and to neighboring slices in the stack. To this end, we formulate two potential functions and associate them to the same Markov random field through which we can efficiently find an optimal solution. Due to our simultaneous formulation and the absence of any segmentation step during the reconstruction we can dramatically reduce error propagation effects. This is illustrated by experiments on carefully created synthetic as well as real data sets.

Selective image similarity measure for bronchoscope tracking based on image registration.

We propose a selective method of measurement for computing image similarities based on characteristic structure extraction and demonstrate its application to flexible endoscope navigation, in particular to a bronchoscope navigation system. Camera motion tracking is a fundamental function required for image-guided treatment or therapy systems. In recent years, an ultra-tiny electromagnetic sensor commercially became available, and many image-guided treatment or therapy systems use this sensor for tracking the camera position and orientation. However, due to space limitations, it is difficult to equip the tip of a bronchoscope with such a position sensor, especially in the case of ultra-thin bronchoscopes. Therefore, continuous image registration between real and virtual bronchoscopic images becomes an efficient tool for tracking the bronchoscope. Usually, image registration is done by calculating the image similarity between real and virtual bronchoscopic images. Since global schemes to measure image similarity, such as mutual information, squared gray-level difference, or cross correlation, average differences in intensity values over an entire region, they fail at tracking of scenes where less characteristic structures can be observed. The proposed method divides an entire image into a set of small subblocks and only selects those in which characteristic shapes are observed. Then image similarity is calculated within the selected subblocks. Selection is done by calculating feature values within each subblock. We applied our proposed method to eight pairs of chest X-ray CT images and bronchoscopic video images. The experimental results revealed that bronchoscope tracking using the proposed method could track up to 1600 consecutive bronchoscopic images (about 50s) without external position sensors. Tracking performance was greatly improved in comparison with a standard method utilizing squared gray-level differences of the entire images.

The virtual mirror: a new interaction paradigm for augmented reality environments.

Medical augmented reality (AR) has been widely discussed within the medical imaging as well as computer aided surgery communities. Different systems for exemplary medical applications have been proposed. Some of them produced promising results. One major issue still hindering AR technology to be regularly used in medical applications is the interaction between physician and the superimposed 3-D virtual data. Classical interaction paradigms, for instance with keyboard and mouse, to interact with visualized medical 3-D imaging data are not adequate for an AR environment. This paper introduces the concept of a tangible/controllable Virtual Mirror for medical AR applications. This concept intuitively augments the direct view of the surgeon with all desired views on volumetric medical imaging data registered with the operation site without moving around the operating table or displacing the patient. We selected two medical procedures to demonstrate and evaluate the potentials of the Virtual Mirror for the surgical workflow. Results confirm the intuitiveness of this new paradigm and its perceptive advantages for AR-based computer aided interventions.

Magneto-optical tracking of flexible laparoscopic ultrasound: model-based online detection and correction of magnetic tracking errors.

Electromagnetic tracking is currently one of the most promising means of localizing flexible endoscopic instruments such as flexible laparoscopic ultrasound transducers. However, electromagnetic tracking is also susceptible to interference from ferromagnetic material, which distorts the magnetic field and leads to tracking errors. This paper presents new methods for real-time online detection and reduction of dynamic electromagnetic tracking errors when localizing a flexible laparoscopic ultrasound transducer. We use a hybrid tracking setup to combine optical tracking of the transducer shaft and electromagnetic tracking of the flexible transducer tip. A novel approach of modeling the poses of the transducer tip in relation to the transducer shaft allows us to reliably detect and significantly reduce electromagnetic tracking errors. For detecting errors of more than 5 mm, we achieved a sensitivity and specificity of 91% and 93%, respectively. Initial 3-D rms error of 6.91 mm were reduced to 3.15 mm.

New approaches to online estimation of electromagnetic tracking errors for laparoscopic ultrasonography.

In abdominal surgery, a laparoscopic ultrasound transducer is commonly used to detect lesions such as metastases. The determination and visualization of the position and orientation of its flexible tip in relation to the patient or other surgical instruments can be a great support for surgeons using the transducer intraoperatively. This difficult subject has recently received attention from the scientific community. Electromagnetic tracking systems can be applied to track the flexible tip; however, current limitations of electromagnetic tracking include its accuracy and sensibility, i.e., the magnetic field can be distorted by ferromagnetic material. This paper presents two novel methods for estimation of electromagnetic tracking error. Based on optical tracking of the laparoscope, as well as on magneto-optic and visual tracking of the transducer, these methods automatically detect in 85% of all cases whether tracking is erroneous or not, and reduce tracking errors by up to 2.5 mm.

Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection.

In recent years, an increasing number of liver tumor indications were treated by minimally invasive laparoscopic resection. Besides the restricted view, two major intraoperative issues in laparoscopic liver resection are the optimal planning of ports as well as the enhanced visualization of (hidden) vessels, which supply the tumorous liver segment and thus need to be divided (e.g., clipped) prior to the resection. We propose an intuitive and precise method to plan the placement of ports. Preoperatively, self-adhesive fiducials are affixed to the patient's skin and a computed tomography (CT) data set is acquired while contrasting the liver vessels. Immediately prior to the intervention, the laparoscope is moved around these fiducials, which are automatically reconstructed to register the patient to its preoperative imaging data set. This enables the simulation of a camera flight through the patient's interior along the laparoscope's or instruments' axes to easily validate potential ports. Intraoperatively, surgeons need to update their surgical planning based on actual patient data after organ deformations mainly caused by application of carbon dioxide pneumoperitoneum. Therefore, preoperative imaging data can hardly be used. Instead, we propose to use an optically tracked mobile C-arm providing cone-beam CT imaging capability intraoperatively. After patient positioning, port placement, and carbon dioxide insufflation, the liver vessels are contrasted and a 3-D volume is reconstructed during patient exhalation. Without any further need for patient registration, the reconstructed volume can be directly augmented on the live laparoscope video, since prior calibration enables both the volume and the laparoscope to be positioned and oriented in the tracking coordinate frame. The augmentation provides the surgeon with advanced visual aid for the localization of veins, arteries, and bile ducts to be divided or sealed.

Navigation in endoscopic soft tissue surgery: perspectives and limitations.

Despite rapid developments in the research areas of medical imaging, medical image processing, and robotics, the use of computer assistance in surgical routine is still limited to diagnostics, surgical planning, and interventions on mostly rigid structures. In order to establish a computer-aided workflow from diagnosis to surgical treatment and follow-up, several proposals for computer-assisted soft tissue interventions have been made in recent years. By means of different pre- and intraoperative information sources, such as surgical planning, intraoperative imaging, and tracking devices, surgical navigation systems aim to support surgeons in localizing anatomical targets, observing critical structures, and sparing healthy tissue. Current research in particular addresses the problem of organ shift and tissue deformation, and obstacles in communication between navigation system and surgeon. In this paper, we review computer-assisted navigation systems for soft tissue surgery. We concentrate on approaches that can be applied in endoscopic thoracic and abdominal surgery, because endoscopic surgery has special needs for image guidance due to limitations in perception. Furthermore, this paper informs the reader about new trends and technologies in the area of computer-assisted surgery. Finally, a balancing of the key challenges and possible benefits of endoscopic navigation refines the perspectives of this increasingly important discipline of computer-aided medical procedures.

Magneto-optic tracking of a flexible laparoscopic ultrasound transducer for laparoscope augmentation.

In abdominal surgery, a laparoscopic ultrasound transducer is commonly used to detect lesions such as metastases. The determination and visualization of position and orientation of its flexible tip in relation to the patient or other surgical instruments can be of much help to (novice) surgeons utilizing the transducer intraoperatively. This difficult subject has recently been paid attention to by the scientific community . Electromagnetic tracking systems can be applied to track the flexible tip. However, the magnetic field can be distorted by ferromagnetic material. This paper presents a new method based on optical tracking of the laparoscope and magneto-optic tracking of the transducer, which is able to automatically detect field distortions. This is used for a smooth augmentation of the B-scan images of the transducer directly on the camera images in real time.

Real-time fusion of ultrasound and gamma probe for navigated localization of liver metastases.

Liver metastases are an advanced stage of several types of cancer, usually treated with surgery. Intra-operative localization of these lesions is currently facilitated by intra-operative ultrasound (IOUS) and palpation, yielding a high rate of false positives due to benign abnormal regions. In this paper we present the integration of functional nuclear information from a gamma probe with IOUS, to provide a synchronized, real-time visualization that facilitates the detection of active metastases intra-operatively. We evaluate the system in an ex-vivo setup employing a group of physicians and medical technicians and show that the addition of functional imaging improves the accuracy of localizing and identifying malignant and benign lesions significantly. Furthermore we are able to demonstrate that the inclusion of an advanced, augmented visualization provides more reliability and confidence on classifying these lesions in the presented evaluation setup.

Automatic patient registration for port placement in minimally invasive endoscopic surgery.

Optimal port placement is a delicate issue in minimally invasive endoscopic surgery, particularly in robotically assisted surgery. A good choice of the instruments' and endoscope's ports can avoid time-consuming consecutive new port placement. We present a novel method to intuitively and precisely plan the port placement. The patient is registered to its pre-operative CT by just moving the endoscope around fiducials, which are attached to the patient's thorax and are visible in its CT. Their 3D positions are automatically reconstructed. Without prior time-consuming segmentation, the pre-operative CT volume is directly rendered with respect to the endoscope or instruments. This enables the simulation of a camera flight through the patient's interior along the instruments' axes to easily validate possible ports.