Shape Sensing Techniques for Continuum Robots in Minimally Invasive Surgery: A Survey.

Continuum robots provide inherent structural compliance with high dexterity to access the surgical target sites along tortuous anatomical paths under constrained environments and enable to perform complex and delicate operations through small incisions in minimally invasive surgery. These advantages enable their broad applications with minimal trauma and make challenging clinical procedures possible with miniaturized instrumentation and high curvilinear access capabilities. However, their inherent deformable designs make it difficult to realize 3-D intraoperative real-time shape sensing to accurately model their shape. Solutions to this limitation can lead themselves to further develop closely associated techniques of closed-loop control, path planning, human-robot interaction, and surgical manipulation safety concerns in minimally invasive surgery. Although extensive model-based research that relies on kinematics and mechanics has been performed, accurate shape sensing of continuum robots remains challenging, particularly in cases of unknown and dynamic payloads. This survey investigates the recent advances in alternative emerging techniques for 3-D shape sensing in this field and focuses on the following categories: fiber-optic-sensor-based, electromagnetic-tracking-based, and intraoperative imaging modality-based shape-reconstruction methods. The limitations of existing technologies and prospects of new technologies are also discussed.

Robust electromagnetically guided endoscopic procedure using enhanced particle swarm optimization for multimodal information fusion.

PURPOSE: Electromagnetically guided endoscopic procedure, which aims at accurately and robustly localizing the endoscope, involves multimodal sensory information during interventions. However, it still remains challenging in how to integrate these information for precise and stable endoscopic guidance. To tackle such a challenge, this paper proposes a new framework on the basis of an enhanced particle swarm optimization method to effectively fuse these information for accurate and continuous endoscope localization. METHODS: The authors use the particle swarm optimization method, which is one of stochastic evolutionary computation algorithms, to effectively fuse the multimodal information including preoperative information (i.e., computed tomography images) as a frame of reference, endoscopic camera videos, and positional sensor measurements (i.e., electromagnetic sensor outputs). Since the evolutionary computation method usually limits its possible premature convergence and evolutionary factors, the authors introduce the current (endoscopic camera and electromagnetic sensor's) observation to boost the particle swarm optimization and also adaptively update evolutionary parameters in accordance with spatial constraints and the current observation, resulting in advantageous performance in the enhanced algorithm. RESULTS: The experimental results demonstrate that the authors' proposed method provides a more accurate and robust endoscopic guidance framework than state-of-the-art methods. The average guidance accuracy of the authors' framework was about 3.0 mm and 5.6° while the previous methods show at least 3.9 mm and 7.0°. The average position and orientation smoothness of their method was 1.0 mm and 1.6°, which is significantly better than the other methods at least with (2.0 mm and 2.6°). Additionally, the average visual quality of the endoscopic guidance was improved to 0.29. CONCLUSIONS: A robust electromagnetically guided endoscopy framework was proposed on the basis of an enhanced particle swarm optimization method with using the current observation information and adaptive evolutionary factors. The authors proposed framework greatly reduced the guidance errors from (4.3, 7.8) to (3.0 mm, 5.6°), compared to state-of-the-art methods.

Robust bronchoscope motion tracking using sequential Monte Carlo methods in navigated bronchoscopy: dynamic phantom and patient validation.

PURPOSE: Accurate and robust estimates of camera position and orientation in a bronchoscope are required for navigation. Fusion of pre-interventional information (e.g., CT, MRI, or US) and intra-interventional information (e.g., bronchoscopic video) were incorporated into a navigation system to provide physicians with an augmented reality environment for bronchoscopic interventions. METHODS: Two approaches were used to predict bronchoscope movements by incorporating sequential Monte Carlo (SMC) simulation including (1) image-based tracking techniques and (2) electromagnetic tracking (EMT) methods. SMC simulation was introduced to model ambiguities or uncertainties that occurred in image- and EMT-based bronchoscope tracking. Scale invariant feature transform (SIFT) features were employed to overcome the limitations of image-based motion tracking methods. Validation was performed on five phantom and ten human case datasets acquired in the supine position. RESULTS: For dynamic phantom validation, the EMT-SMC simulation method improved the tracking performance of the successfully registered bronchoscopic video frames by 12.7% compared with a hybrid-based method. In comparisons between tracking results and ground truth, the accuracy of the EMT-SMC simulation method was 1.51 mm (positional error) and 5.44° (orientation error). During patient assessment, the SIFT-SMC simulation scheme was more stable or robust than a previous image-based approach for bronchoscope motion estimation, showing 23.6% improvement of successfully tracked frames. Comparing the estimates of our method to ground truth, the position and orientation errors are 3.72 mm and 10.2°, while those of our previous image-based method were at least 7.77 mm and 19.3°. The computational times of our EMT- and SIFT-SMC simulation methods were 0.9 and 1.2 s per frame, respectively. CONCLUSION: The SMC simulation method was developed to model ambiguities that occur in bronchoscope tracking. This method more stably and accurately predicts the bronchoscope camera position and orientation parameters, reducing uncertainties due to problematic bronchoscopic video frames and airway deformation during intra-bronchoscopy navigation.

Bronchoscopy navigation beyond electromagnetic tracking systems: a novel bronchoscope tracking prototype.

A novel bronchoscope tracking prototype was designed and validated for bronchoscopic navigation. We construct a novel mouth- or nose-piece bronchoscope model to directly measure the movement information of a bronchoscope outside of a patient's body. Fusing the measured movement information based on sequential Monte Carlo (SMC) sampler, we exploit accurate and robust intra-operative alignment between the pre- and intra-operative image data for augmenting surgical bronchoscopy. We validate our new prototype on phantom datasets. The experimental results demonstrate that our proposed prototype is a promising approach to navigate a bronchoscope beyond EMT systems.

ManiSMC: a new method using manifold modeling and sequential Monte Carlo sampler for boosting navigated bronchoscopy.

This paper presents a new bronchoscope motion tracking method that utilizes manifold modeling and sequential Monte Carlo (SMC) sampler to boost navigated bronchoscopy. Our strategy to estimate the bronchoscope motions comprises two main stages: (1) bronchoscopic scene identification and (2) SMC sampling. We extend a spatial local and global regressive mapping (LGRM) method to Spatial-LGRM to learn bronchoscopic video sequences and construct their manifolds. By these manifolds, we can classify bronchoscopic scenes to bronchial branches where a bronchoscope is located. Next, we employ a SMC sampler based on a selective image similarity measure to integrate estimates of stage (1) to refine positions and orientations of a bronchoscope. Our proposed method was validated on patient datasets. Experimental results demonstrate the effectiveness and robustness of our method for bronchoscopic navigation without an additional position sensor.