Target-oriented deep learning-based image registration with individualized test-time adaptation.

BACKGROUND: A classic approach in medical image registration is to formulate an optimization problem based on the image pair of interest, and seek a deformation vector field (DVF) to minimize the corresponding objective, often iteratively. It has a clear focus on the targeted pair, but is typically slow. In contrast, more recent deep-learning-based registration offers a much faster alternative and can benefit from data-driven regularization. However, learning is a process to "fit" the training cohort, whose image or motion characteristics or both may differ from the pair of images to be tested, which is the ultimate goal of registration. Therefore, generalization gap poses a high risk with direct inference alone. PURPOSE: In this study, we propose an individualized adaptation to improve test sample targeting, to achieve a synergy of efficiency and performance in registration. METHODS: Using a previously developed network with an integrated motion representation prior module as the implementation backbone, we propose to adapt the trained registration network further for image pairs at test time to optimize the individualized performance. The adaptation method was tested against various characteristics shifts caused by cross-protocol, cross-platform, and cross-modality, with test evaluation performed on lung CBCT, cardiac MRI, and lung MRI, respectively. RESULTS: Landmark-based registration errors and motion-compensated image enhancement results demonstrated significantly improved test registration performance from our method, compared to tuned classic B-spline registration and network solutions without adaptation. CONCLUSIONS: We have developed a method to synergistically combine the effectiveness of pre-trained deep network and the target-centric perspective of optimization-based registration to improve performance on individual test data.

A conditional registration network for continuous 4D respiratory motion synthesis.

BACKGROUND: Four-dimensional computed tomography (4DCT) provides an important physiological information for diagnosis and treatment. On the other hand, its acquisition could be challenged by artifacts due to motion sorting/binning, time and effort bandwidth in image quality QA, and dose considerations. A 4D synthesis development would significantly augment the available data, addressing quality and consistency issues. Furthermore, the high-quality synthesis can serve as an essential backbone to establish a feasible physiological manifold to support online reconstruction, registration, and downstream analysis from real-time x-ray imaging. PURPOSE: Our study aims to synthesize continuous 4D respiratory motion from two extreme respiration phases. METHODS: A conditional image registration network is trained to take the end-inhalation (EI) and end-exhalation (EE) as input, and output arbitrary breathing phases by varying the conditional variable. A volume compensation and calibration post-processing is further introduced to improve intensity synthesis accuracy. The method was tested on 20 4DCT scans with a four-fold cross-testing scheme and compared against two linear scaling methods and an image translation network. RESULTS: Our method generated realistic 4D respiratory motion fields that were spatiotemporally smooth, achieving a root-mean-square error of (70.1 ± 33.0) HU and structural similarity index of (0.926 ± 0.044), compared to the ground-truth 4DCT. A 10-phase synthesis takes about 2.85 s. CONCLUSIONS: We have presented a novel paradigm to synthesize continuous 4D respiratory motion from end-inhale and end-exhale image pair. By varying the conditional variable, the network can generate the motion field for an arbitrary intermediate breathing phase with precise control.

MRA-free intracranial vessel localization on MR vessel wall images.

Analysis of vessel morphology is important in assessing intracranial atherosclerosis disease (ICAD). Recently, magnetic resonance (MR) vessel wall imaging (VWI) has been introduced to image ICAD and characterize morphology for atherosclerotic lesions. In order to automatically perform quantitative analysis on VWI data, MR angiography (MRA) acquired in the same imaging session is typically used to localize the vessel segments of interest. However, MRA may be unavailable caused by the lack or failure of the sequence in a VWI protocol. This study aims to investigate the feasibility to infer the vessel location directly from VWI. We propose to synergize an atlas-based method to preserve general vessel structure topology with a deep learning network in the motion field domain to correct the residual geometric error. Performance is quantified by examining the agreement between the extracted vessel structures from the pair-acquired and alignment-corrected angiogram, and the estimated output using a cross-validation scheme. Our proposed pipeline yields clinically feasible performance in localizing intracranial vessels, demonstrating the promise of performing vessel morphology analysis using VWI alone.

Inter-Phase 4D Cardiac MRI Registration With a Motion Prior Derived From CTA.

OBJECTIVE: Registration between phases in 4D cardiac MRI is essential for reconstructing high-quality images and appreciating the dynamics. Complex motion and limited image quality make it challenging to design regularization functionals. We propose to introduce a motion representation model (MRM) into a registration network to impose customized, site-specific, and spatially variant prior for cardiac motion. METHODS: We propose a novel approach to regularize deep registration with a deformation vextor field (DVF) representation model using computed tomography angiography (CTA). In the form of a convolutional auto-encoder, the MRM was trained to capture the spatially variant pattern of feasible DVF Jacobian. The CTA-derived MRM was then incorporated into an unsupervised network to facilitate MRI registration. In the experiment, 10 CTAs were used to derive the MRM. The method was tested on 10 0.35 T scans in long-axis view with manual segmentation and 15 3 T scans in short-axis view with tagging-based landmarks. RESULTS: Introducing the MRM improved registration accuracy and achieved 2.23, 7.21, and 4.42 mm 80% Hausdorff distance on left ventricle, right ventricle, and pulmonary artery, respectively, and 2.23 mm landmark registration error. The results were comparable to carefully tuned SimpleElastix, but reduced the registration time from 40 s to 0.02 s. The MRM presented good robustness to different DVF sample generation methods. CONCLUSION: The model enjoys high accuracy as meticulously tuned optimization model and the efficiency of deep networks. SIGNIFICANCE: The method enables model to go beyond the quality limitation of MRI. The robustness to training DVF generation scheme makes the method attractive to adapting to the available data and software resources in various clinics.

Scale-adaptive deep network for deformable image registration.

PURPOSE: Multiresolution hierarchical strategy is typically used in conventional optimization-based image registration to capture varying magnitudes of deformations while avoiding undesirable local minima. A rough concept of the scale is captured in deep networks by the reception field of kernels, and it has been realized to be both desirable and challenging to capture convolutions of different scales simultaneously in registration networks. In this study, we propose a registration network that is conscious of and self-adaptive to deformation of various scales to improve registration performance. METHODS: Dilated inception modules (DIMs) are proposed to incorporate receptive fields of different sizes in a computationally efficient way. Scale adaptive modules (SAMs) are proposed to guide and adjust shallow features using convolutional kernels with spatially adaptive dilation rate learned from deep features. DIMs and SAMs are integrated into a registration network which takes a U-net structure. The network is trained in an unsupervised setting and completes registration with a single evaluation run. RESULTS: Experiment with two-dimensional (2D) cardiac MRIs showed that the adaptive dilation rate in SAM corresponded well to the deformation scale. Evaluated with left ventricle segmentation, our method achieved a Dice coefficient of (0.93 ± 0.02), significantly better than SimpleElastix and networks without DIM or SAM. The average surface distance was less than 2 mm, comparable to SimpleElastix without statistical significance. Experiment with synthetic data demonstrated the effectiveness of DIMs and SAMs, which led to a significant reduction in target registration error (TRE) based on dense deformation field. The three-dimensional (3D) version of the network achieved a 2.52 mm mean TRE on anatomical landmarks in DIR-Lab thoracic 4DCTs, lower than SimpleElastix and networks without DIM or SAM with statistical significance. The average registration times were 0.002 s for 2D images with size 256 × 256 and 0.42 s for 3D images with size 256 × 256 × 96. CONCLUSIONS: The introduction and integration of DIMs and SAMs addressed the heterogeneous scale problem in an efficient and self-adaptive way. The proposed method provides an alternative to the inefficient multiresolution registration setups.

Imposing implicit feasibility constraints on deformable image registration using a statistical generative model.

Purpose: Deformable registration problems are conventionally posed in a regularized optimization framework, where balance between fidelity and prescribed regularization usually needs to be tuned for each case. Even so, using a single weight to control regularization strength may be insufficient to reflect spatially variant tissue properties and limit registration performance. In this study, we proposed to incorporate a spatially variant deformation prior into image registration framework using a statistical generative model. Approach: A generator network is trained in an unsupervised setting to maximize the likelihood of observing the moving and fixed image pairs, using an alternating back-propagation approach. The trained model imposes constraints on deformation and serves as an effective low-dimensional deformation parametrization. During registration, optimization is performed over this learned parametrization, eliminating the need for explicit regularization and tuning. The proposed method was tested against SimpleElastix, DIRNet, and Voxelmorph. Results: Experiments with synthetic images and simulated CTs showed that our method yielded registration errors significantly lower than SimpleElastix and DIRNet. Experiments with cardiac magnetic resonance images showed that the method encouraged physical and physiological feasibility of deformation. Evaluation with left ventricle contours showed that our method achieved a dice of ( 0.93 ± 0.03 ) with significant improvement over all SimpleElastix options, DIRNet, and VoxelMorph. Mean average surface distance was on millimeter level, comparable to the best SimpleElastix setting. The average 3D registration time was 12.78 s, faster than 24.70 s in SimpleElastix. Conclusions: The learned implicit parametrization could be an efficacious alternative to regularized B-spline model, more flexible in admitting spatial heterogeneity.