Development and Pre-Clinical Analysis of Spatiotemporal-Aware Augmented Reality in Orthopedic Interventions.

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. As a consequence, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. The main contribution of this paper is to reveal how exemplary workflows are redefined by taking full advantage of head-mounted displays when entirely co-registered with the imaging system at all times. The awareness of the system from the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. Our system achieved an error of 4.76 ± 2.91mm for placing K-wire in a fracture management procedure, and yielded errors of 1.57 ± 1.16° and 1.46 ± 1.00° in the abduction and anteversion angles, respectively, for total hip arthroplasty (THA). We compared the results with the outcomes from baseline standard operative and non-immersive AR procedures, which had yielded errors of [4.61mm, 4.76°, 4.77°] and [5.13mm, 1.78°, 1.43°], respectively, for wire placement, and abduction and anteversion during THA. We hope that our holistic approach towards improving the interface of surgery not only augments the surgeon's capabilities but also augments the surgical team's experience in carrying out an effective intervention with reduced complications and provide novel approaches of documenting procedures for training purposes.

Robust navigation support in lowest dose image setting.

PURPOSE: Clinical cardiac electrophysiology (EP) is concerned with diagnosis and treatment of cardiac arrhythmia describing abnormality or perturbation in the normal activation sequence of the myocardium. With the recent introduction of lowest dose X-ray imaging protocol for EP procedures, interventional image enhancement has gained crucial importance for the well-being of patients as well as medical staff. METHODS: In this paper, we introduce a novel method to detect and track different EP catheter electrodes in lowest dose fluoroscopic sequences based on [Formula: see text]-sparse coding and online robust PCA (ORPCA). Besides being able to work on real lowest dose sequences, the underlying methodology achieves simultaneous detection and tracking of three main EP catheters used during ablation procedures. RESULTS: We have validated our algorithm on 16 lowest dose fluoroscopic sequences acquired during real cardiac ablation procedures. In addition to expert labels for 2 sequences, we have employed a crowdsourcing strategy to obtain ground truth labels for the remaining 14 sequences. In order to validate the effect of different training data, we have employed a leave-one-out cross-validation scheme yielding an average detection rate of [Formula: see text]. CONCLUSION: Besides these promising quantitative results, our medical partners also expressed their high satisfaction. Being based on [Formula: see text]-sparse coding and online robust PCA (ORPCA), our method advances previous approaches by being able to detect and track electrodes attached to multiple different catheters.

Automatic 3D reconstruction of electrophysiology catheters from two-view monoplane C-arm image sequences.

PURPOSE: Catheter guidance is a vital task for the success of electrophysiology interventions. It is usually provided through fluoroscopic images that are taken intra-operatively. The cardiologists, who are typically equipped with C-arm systems, scan the patient from multiple views rotating the fluoroscope around one of its axes. The resulting sequences allow the cardiologists to build a mental model of the 3D position of the catheters and interest points from the multiple views. METHOD: We describe and compare different 3D catheter reconstruction strategies and ultimately propose a novel and robust method for the automatic reconstruction of 3D catheters in non-synchronized fluoroscopic sequences. This approach does not purely rely on triangulation but incorporates prior knowledge about the catheters. In conjunction with an automatic detection method, we demonstrate the performance of our method compared to ground truth annotations. RESULTS: In our experiments that include 20 biplane datasets, we achieve an average reprojection error of 0.43 mm and an average reconstruction error of 0.67 mm compared to gold standard annotation. CONCLUSIONS: In clinical practice, catheters suffer from complex motion due to the combined effect of heartbeat and respiratory motion. As a result, any 3D reconstruction algorithm via triangulation is imprecise. We have proposed a new method that is fully automatic and highly accurate to reconstruct catheters in three dimensions.

Multi-scale tubular structure detection in ultrasound imaging.

We propose a novel, physics-based method for detecting multi-scale tubular features in ultrasound images. The detector is based on a Hessian-matrix eigenvalue method, but unlike previous work, our detector is guided by an optimal model of vessel-like structures with respect to the ultrasound-image formation process. Our method provides a voxel-wise probability map, along with estimates of the radii and orientations of the detected tubes. These results can then be used for further processing, including segmentation and enhanced volume visualization. Most Hessian-based algorithms, including the well-known Frangi filter, were developed for CTA or MRA; they implicitly assume symmetry about the vessel centerline. This is not consistent with ultrasound data. We overcome this limitation by introducing a novel filter that allows multi-scale estimation both with respect to the vessel's centerline and with respect to the vessel's border. We use manually-segmented ultrasound imagery from 35 patients to show that our method is superior to standard Hessian-based methods. We evaluate the performance of the proposed methods based on the sensitivity and specificity like measures, and finally demonstrate further applicability of our method to vascular ultrasound images of the carotid artery, as well as ultrasound data for abdominal aortic aneurysms.

Automatic detection of multiple and overlapping EP catheters in fluoroscopic sequences.

We propose a method to perform automatic detection of electrophysiology (EP) catheters in fluoroscopic sequences. Our approach does not need any initialization, is completely automatic, and can detect an arbitrary number of catheters at the same time. The method is based on the usage of blob detectors and clustering in order to detect all catheter electrodes, overlapping or not, within the X-ray images. The proposed technique is validated on 1422 fluoroscopic images yielding a tip detection rate of 99.3% and mean distance of 0.5mm from manually labeled ground truth centroids for all electrodes.

1D-3D registration for functional nuclear imaging.

Functional nuclear imaging systems like PET or SPECT provide unique information that is used extensively in diagnosis, but it has also proven very useful for image-guided interventions. In the case of SPECT and radio-guided surgeries, 1D gamma detectors called gamma probes are routinely used during interventions to localize hotspots in conjunction with pre-operative SPECT images, or more recently, intraoperative SPECT images. As the tissue is being manipulated during surgery, these SPECT images quickly lose their validity, necessitating either new scans, which is in most cases unfeasible, or requiring the surgeon to do a mental update of the available imagery. In this paper, we present a novel 1D-3D registration procedure for functional nuclear imaging that registers tracked intra-operative 1D probe readings to a pre- or intra-operatively acquired 3D functional image. This procedure allows incorporating prior knowledge during radio-guided surgeries, enabling rapid updates of the visualization in the case of tissue deformation without the overhead of an additional complete scan. We show results using phantom data as well as patient data.

Patient-specific modeling and quantification of the aortic and mitral valves from 4-D cardiac CT and TEE.

As decisions in cardiology increasingly rely on noninvasive methods, fast and precise image processing tools have become a crucial component of the analysis workflow. To the best of our knowledge, we propose the first automatic system for patient-specific modeling and quantification of the left heart valves, which operates on cardiac computed tomography (CT) and transesophageal echocardiogram (TEE) data. Robust algorithms, based on recent advances in discriminative learning, are used to estimate patient-specific parameters from sequences of volumes covering an entire cardiac cycle. A novel physiological model of the aortic and mitral valves is introduced, which captures complex morphologic, dynamic, and pathologic variations. This holistic representation is hierarchically defined on three abstraction levels: global location and rigid motion model, nonrigid landmark motion model, and comprehensive aortic-mitral model. First we compute the rough location and cardiac motion applying marginal space learning. The rapid and complex motion of the valves, represented by anatomical landmarks, is estimated using a novel trajectory spectrum learning algorithm. The obtained landmark model guides the fitting of the full physiological valve model, which is locally refined through learned boundary detectors. Measurements efficiently computed from the aortic-mitral representation support an effective morphological and functional clinical evaluation. Extensive experiments on a heterogeneous data set, cumulated to 1516 TEE volumes from 65 4-D TEE sequences and 690 cardiac CT volumes from 69 4-D CT sequences, demonstrated a speed of 4.8 seconds per volume and average accuracy of 1.45 mm with respect to expert defined ground-truth. Additional clinical validations prove the quantification precision to be in the range of inter-user variability. To the best of our knowledge this is the first time a patient-specific model of the aortic and mitral valves is automatically estimated from volumetric sequences.

Personalized modeling and assessment of the aortic-mitral coupling from 4D TEE and CT.

The anatomy, function and hemodynamics of the aortic and mitral valves are known to be strongly interconnected. An integrated quantitative and visual assessment of the aortic-mitral coupling may have an impact on patient evaluation, planning and guidance of minimal invasive procedures. In this paper, we propose a novel model-driven method for functional and morphological characterization of the entire aortic-mitral apparatus. A holistic physiological model is hierarchically defined to represent the anatomy and motion of the two left heart valves. Robust learning-based algorithms are applied to estimate the patient-specific spatial-temporal parameters from four-dimensional TEE and CT data. The piecewise affine location of the valves is initially determined over the whole cardiac cycle using an incremental search performed in marginal spaces. Consequently, efficient spectrum detection in the trajectory space is applied to estimate the cyclic motion of the articulated model. Finally, the full personalized surface model of the aortic-mitral coupling is constructed using statistical shape models and local spatial-temporal refinement. Experiments performed on 65 4D TEE and 69 4D CT sequences demonstrated an average accuracy of 1.45 mm and speed of 60 seconds for the proposed approach. Initial clinical validation on model-based and expert measurement showed the precision to be in the range of the inter-user variability. To the best of our knowledge this is the first time a complete model of the aortic-mitral coupling estimated from TEE and CT data is proposed.