Non-rigid MR-TRUS image registration for image-guided prostate biopsy using correlation ratio-based mutual information.

BACKGROUND: To improve the accuracy of ultrasound-guided biopsy of the prostate, the non-rigid registration of magnetic resonance (MR) images onto transrectal ultrasound (TRUS) images has gained increasing attention. Mutual information (MI) is a widely used similarity criterion in MR-TRUS image registration. However, the use of MI has been challenged because of intensity distortion, noise and down-sampling. Hence, we need to improve the MI measure to get better registration effect. METHODS: We present a novel two-dimensional non-rigid MR-TRUS registration algorithm that uses correlation ratio-based mutual information (CRMI) as the similarity criterion. CRMI includes a functional mapping of intensity values on the basis of a generalized version of intensity class correspondence. We also analytically acquire the derivative of CRMI with respect to deformation parameters. Furthermore, we propose an improved stochastic gradient descent (ISGD) optimization method based on the Metropolis acceptance criteria to improve the global optimization ability and decrease the registration time. RESULTS: The performance of the proposed method is tested on synthetic images and 12 pairs of clinical prostate TRUS and MR images. By comparing label map registration frame (LMRF) and conditional mutual information (CMI), the proposed algorithm has a significant improvement in the average values of Hausdorff distance and target registration error. Although the average Dice Similarity coefficient is not significantly better than CMI, it still has a crucial increase over LMRF. The average computation time consumed by the proposed method is similar to LMRF, which is 16 times less than CMI. CONCLUSION: With more accurate matching performance and lower sensitivity to noise and down-sampling, the proposed algorithm of minimizing CRMI by ISGD is more robust and has the potential for use in aligning TRUS and MR images for needle biopsy.

A monocular SLAM system based on SIFT features for gastroscope tracking.

During flexible gastroscopy, physicians have extreme difficulties to self-localize. Camera tracking method such as simultaneous localization and mapping (SLAM) has become a research hotspot in recent years, allowing tracking of the endoscope. However, most of the existing solutions have focused on tasks in which sufficient texture information is available, such as laparoscope tracking, and cannot be applied to gastroscope tracking since gastroscopic images have fewer textures than laparoscopic images. This paper proposes a new monocular SLAM framework based on scale-invariant feature transform (SIFT) and narrow-band imaging (NBI), which extracts SIFT features instead of oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (BRIEF) features from gastroscopic NBI images, and performs feature retention based on the response sorting strategy for achieving more matches. Experimental results show that the root mean squared error of the proposed algorithm can reach a minimum of 2.074 mm, and the pose accuracy can be improved by up to 25.73% compared with oriented FAST and rotated BRIEF (ORB)-SLAM. SIFT features and response sorting strategy can achieve more accurate matching in gastroscopic NBI images than other features and homogenization strategy, and the proposed algorithm can also run successfully on real clinical gastroscopic data. The proposed algorithm has the potential clinical value to assist physicians in locating the gastroscope during gastroscopy.

Intensity-based nonrigid endomicroscopic image mosaicking incorporating texture relevance for compensation of tissue deformation.

Image mosaicking has emerged as a universal technique to broaden the field-of-view of the probe-based confocal laser endomicroscopy (pCLE) imaging system. However, due to the influence of probe-tissue contact forces and optical components on imaging quality, existing mosaicking methods remain insufficient to deal with practical challenges. In this paper, we present the texture encoded sum of conditional variance (TESCV) as a novel similarity metric, and effectively incorporate it into a sequential mosaicking scheme to simultaneously correct rigid probe shift and nonrigid tissue deformation. TESCV combines both intensity dependency and texture relevance to quantify the differences between pCLE image frames, where a discriminative binary descriptor named fully cross-detected local derivative pattern (FCLDP) is designed to extract more detailed structural textures. Furthermore, we also analytically derive the closed-form gradient of TESCV with respect to the transformation variables. Experiments on the circular dataset highlighted the advantage of the TESCV metric in improving mosaicking performance compared with the other four recently published metrics. The comparison with the other four state-of-the-art mosaicking methods on the spiral and manual datasets indicated that the proposed TESCV-based method not only worked stably at different contact forces, but was also suitable for both low- and high-resolution imaging systems. With more accurate and delicate mosaics, the proposed method holds promises to meet clinical demands for intraoperative optical biopsy.

CDFRegNet: A cross-domain fusion registration network for CT-to-CBCT image registration.

BACKGROUND AND OBJECTIVE: Computer tomography (CT) to cone-beam computed tomography (CBCT) image registration plays an important role in radiotherapy treatment placement, dose verification, and anatomic changes monitoring during radiotherapy. However, fast and accurate CT-to-CBCT image registration is still very challenging due to the intensity differences, the poor image quality of CBCT images, and inconsistent structure information. METHODS: To address these problems, a novel unsupervised network named cross-domain fusion registration network (CDFRegNet) is proposed. First, a novel edge-guided attention module (EGAM) is designed, aiming at capturing edge information based on the gradient prior images and guiding the network to model the spatial correspondence between two image domains. Moreover, a novel cross-domain attention module (CDAM) is proposed to improve the network's ability to guide the network to effectively map and fuse the domain-specific features. RESULTS: Extensive experiments on a real clinical dataset were carried out, and the experimental results verify that the proposed CDFRegNet can register CT to CBCT images effectively and obtain the best performance, while compared with other representative methods, with a mean DSC of 80.01±7.16%, a mean TRE of 2.27±0.62 mm, and a mean MHD of 1.50±0.32 mm. The ablation experiments also proved that our EGAM and CDAM can further improve the accuracy of the registration network and they can generalize well to other registration networks. CONCLUSION: This paper proposed a novel CT-to-CBCT registration method based on EGAM and CDAM, which has the potential to improve the accuracy of multi-domain image registration.

Miniature Flexible Instrument with Fibre Bragg Grating-Based Triaxial Force Sensing for Intraoperative Gastric Endomicroscopy.

Optical biopsy methods, such as probe-based endomicroscopy, can be used to identify early-stage gastric cancer in vivo. However, it is difficult to scan a large area of the gastric mucosa for mosaicking during endoscopy. In this work, we propose a miniaturised flexible instrument based on contact-aided compliant mechanisms and fibre Bragg grating (FBG) sensing for intraoperative gastric endomicroscopy. The instrument has a compact design with an outer diameter of 2.7 mm, incorporating a central channel with a diameter of 1.9 mm for the endomicroscopic probe to pass through. Experimental results show that the instrument can achieve raster trajectory scanning over a large tissue surface with a positioning accuracy of 0.5 mm. The tip force sensor provides a 4.6 mN resolution for the axial force and 2.8 mN for transverse forces. Validation with random samples shows that the force sensor can provide consistent and accurate three-axis force detection. Endomicroscopic imaging experiments were conducted, and the flexible instrument performed no gap scanning (mosaicking area more than 3 mm2) and contact force monitoring during scanning, demonstrating the potential of the system in clinical applications.

Robust Mosaicing of Endomicroscopic Videos via Context-Weighted Correlation Ratio.

Probe-based confocal laser endomicroscopy (pCLE) is a promising imaging tool that provides in situ and in vivo optical imaging to perform real-time pathological assessments. However, due to limited field of view, it is difficult for clinicians to get a full understanding of the scanned tissues. In this paper, we develop a novel mosaicing framework to assemble all frame sequences into a full view image. First, a hybrid rigid registration that combines feature matching and template matching is presented to achieve a global alignment of all frames. Then, the parametric free-form deformation (FFD) model with a multiresolution architecture is implemented to accommodate non-rigid tissue distortions. More importantly, we devise a robust similarity metric called context-weighted correlation ratio (CWCR) to promote registration accuracy, where spatial and geometric contexts are incorporated into the estimation of functional intensity dependence. Experiments on both robotic setup and manual manipulation have demonstrated that the proposed scheme significantly precedes some state-of-the-art mosaicing schemes in the presence of intensity fluctuations, insufficient overlap and tissue distortions. Moreover, the comparisons of the proposed CWCR metric and two other metrics have validated the effectiveness of the context-weighted strategy in quantifying the differences between two frames. Benefiting from more rational and delicate mosaics, the proposed scheme is more suitable to instruct diagnosis and treatment during optical biopsies.

Hand-Held Instrument with Integrated Parallel Mechanism for Active Tremor Compensation During Microsurgery.

Physiological hand tremor seriously influences the surgical instrument's tip positioning accuracy during microsurgery. To solve this problem, hand-held active tremor compensation instruments are developed to improve tip positioning accuracy during microsurgery. This paper presents the design and performance of a new hand-held instrument that aims to stabilize hand tremors and increase accuracy in microsurgery. The key components are a three degrees of freedom (DOF) integrated parallel manipulator and a high-performance inertial measurement unit (IMU). The IMU was developed to sense the 3-DOF motion of the instrument tip. A customized filter was applied to extract specific hand tremor motion. Then, the instrument was employed to generate the reverse motion simultaneously to reduce tremor motion. Experimental results show that the tremor compensation mechanism is effective. The average RMS reduction ratio of bench test is 56.5% that is a significant tremor reduction ratio. For hand-held test, it has an average RMS reduction ratio of 41.0%. Hence, it could reduce hand tremor magnitudes by 31.7% RMS in 2-DOF.

Locally Adaptive Total p-Variation Regularization for Non-Rigid Image Registration With Sliding Motion.

Due to the complicated thoracic movements which contain both sliding motion occurring at lung surfaces and smooth motion within individual organs, respiratory estimation is still an intrinsically challenging task. In this paper, we propose a novel regularization term called locally adaptive total p-variation (LaTpV) and embed it into a parametric registration framework to accurately recover lung motion. LaTpV originates from a modified Lp-norm constraint (1 < p < 2), where a prior distribution of p modeled by the Dirac-shaped function is constructed to specifically assign different values to voxels. LaTpV adaptively balances the smoothness and discontinuity of the displacement field to encourage an expected sliding interface. Additionally, we also analytically deduce the gradient of the cost function with respect to transformation parameters. To validate the performance of LaTpV, we not only test it on two mono-modal databases including synthetic images and pulmonary computed tomography (CT) images, but also on a more difficult thoracic CT and positron emission tomography (PET) dataset for the first time. For all experiments, both the quantitative and qualitative results indicate that LaTpV significantly surpasses some existing regularizers such as bending energy and parametric total variation. The proposed LaTpV based registration scheme might be more superior for sliding motion correction and more potential for clinical applications such as the diagnosis of pleural mesothelioma and the adjustment of radiotherapy plans.

Unsupervised learning for deformable registration of thoracic CT and cone-beam CT based on multiscale features matching with spatially adaptive weighting.

PURPOSE: Cone-beam computed tomography (CBCT) is a common on-treatment imaging widely used in image-guided radiotherapy. Fast and accurate registration between the on-treatment CBCT and planning CT is significant for and precise adaptive radiotherapy treatment (ART). However, existing CT-CBCT registration methods, which are mostly affine or time-consuming intensity- based deformation registration, still need further study due to the considerable CT-CBCT intensity discrepancy and the artifacts in low-quality CBCT images. In this paper, we propose a deep learning-based CT-CBCT registration model to promote rapid and accurate CT-CBCT registration for radiotherapy. METHODS: The proposed CT-CBCT registration model consists of a registration network and an innovative deep similarity metric network. The registration network is a novel fully convolution network adapted specially for patch-wise CT-CBCT registration. The metric network, going beyond intensity, automatically evaluates the high-dimensional attribute-based dissimilarity between the registered CT and CBCT images. In addition, considering the artifacts in low-quality CBCT images, we add spatial weighting (SW) block to adaptively attach more importance to those informative voxels while inhibit the interference of artifact regions. Such SW-based metric network is expected to extract the most meaningful and discriminative deep features, and form a more reliable CT-CBCT similarity measure to train the registration network. RESULTS: We evaluate the proposed method on clinical thoracic CBCT and CT dataset, and compare the registration results with some other common image similarity metrics and some state-of-the-art registration algorithms. The proposed method provides the highest Structural Similarity index (86.17 ± 5.09), minimum Target Registration Error of landmarks (2.37 ± 0.32 mm), and the best DSC coefficient (78.71 ± 10.95) of tumor volumes. Moreover, our model also obtains comparable distance error of lung surfaces (1.75 ± 0.35 mm). CONCLUSION: The proposed model shows both efficiency and efficacy for reliable thoracic CT-CBCT registration, and can generate the matched CT and CBCT images within few seconds, which is of great significance to clinical radiotherapy.

Nonrigid Image Registration Using Spatially Region-Weighted Correlation Ratio and GPU-Acceleration.

OBJECTIVE: Nonrigid image registration with high accuracy and efficiency remains a challenging task for medical image analysis. In this paper, we present the spatially region-weighted correlation ratio (SRWCR) as a novel similarity measure to improve the registration performance. METHODS: SRWCR is rigorously deduced from a three-dimension joint probability density function combining the intensity channels with an extra spatial information channel. SRWCR estimates the optimal functional dependence between the intensities for each spatial bin, in which the spatial distribution modeled by a cubic B-spline function is used to differentiate the contribution of voxels. We also analytically derive the gradient of SRWCR with respect to the transformation parameters and optimize it using a quasi-Newton approach. Furthermore, we propose a GPU-based parallel mechanism to accelerate the computation of SRWCR and its derivatives. RESULTS: The experiments on synthetic images, public four-dimensional thoracic computed tomography (CT) dataset, retinal optical coherence tomography data, and clinical CT and positron emission tomography images confirm that SRWCR significantly outperforms some state-of-the-art techniques such as spatially encoded mutual information and Robust PaTch-based cOrrelation Ration. CONCLUSION: This study demonstrates the advantages of SRWCR in tackling the practical difficulties due to distinct intensity changes, serious speckle noise, or different imaging modalities. SIGNIFICANCE: The proposed registration framework might be more reliable to correct the nonrigid deformations and more potential for clinical applications.