Neutron tomography and image registration methods to study local physical deformations and attenuation variations in treated archaeological iron nail samples.

This study presents a preliminary examination of the effects of environment changes post-excavation on heavily corroded archaeological Roman iron nails using neutron tomography and image registration techniques. Roman nails were exposed to either a high relative humidity environment, or fast thermal drying as primary experiments to show the power of this imaging technique to monitor and quantify the structural changes of corroded metal artifacts. This research employed a series of pre- and post-treatment tomography acquisitions (time-series) complemented by advanced image registration methods. Based on mutual information (MI) metrics, we performed rigid body and affine image registrations to meticulously account for sample repositioning challenges and variations in imaging parameters. Using non-affine local registration results, in a second step, we detected localized expansion and shrinkage in the samples attributable to imposed environmental changes. Specifically, we observed local shrinkage on the nail that was dried, mostly in their Transformed Medium (TM), the outer layer where corrosion products are cementing soil and sand particles. Conversely, the sample subjected to high relative humidity environment exhibited localized expansion, with varying degrees of change across different regions. This work highlights the efficacy of our registration techniques in accommodating manual removal or loss of extraneous material (loosely adhering soil and TM layers around the nails) post-initial tomography, successfully capturing local structural changes with high precision. Using differential analysis on the accurately registered samples we could also detect and volumetrically quantify the variation in moisture and detect changes in active corrosion sites (ACS) in the sample. These preliminary experiments allowed us to advance and optimize the application of a neutron tomography and image registration workflow for future, more advanced experiments such as humidity fluctuations, corrosion removal through micro-blasting, dechlorination and other stabilization treatments.

A vessel bifurcation liver CT landmark pair dataset for evaluating deformable image registration algorithms.

PURPOSE: Evaluating deformable image registration (DIR) algorithms is vital for enhancing algorithm performance and gaining clinical acceptance. However, there is a notable lack of dependable DIR benchmark datasets for assessing DIR performance except for lung images. To address this gap, we aim to introduce our comprehensive liver computed tomography (CT) DIR landmark dataset library. This dataset is designed for efficient and quantitative evaluation of various DIR methods for liver CTs, paving the way for more accurate and reliable image registration techniques. ACQUISITION AND VALIDATION METHODS: Forty CT liver image pairs were acquired from several publicly available image archives and authors' institutions under institutional review board (IRB) approval. The images were processed with a semi-automatic procedure to generate landmark pairs: (1) for each case, liver vessels were automatically segmented on one image; (2) landmarks were automatically detected at vessel bifurcations; (3) corresponding landmarks in the second image were placed using two deformable image registration methods to avoid algorithm-specific biases; (4) a comprehensive validation process based on quantitative evaluation and manual assessment was applied to reject outliers and ensure the landmarks' positional accuracy. This workflow resulted in an average of ∼56 landmark pairs per image pair, comprising a total of 2220 landmarks for 40 cases. The general landmarking accuracy of this procedure was evaluated using digital phantoms and manual landmark placement. The landmark pair target registration errors (TRE) on digital phantoms were 0.37 ± 0.26 and 0.55 ± 0.34 mm respectively for the two selected DIR algorithms used in our workflow, with 97% of landmark pairs having TREs below 1.5 mm. The distances from the calculated landmarks to the averaged manual placement were 1.27 ± 0.79 mm. DATA FORMAT AND USAGE NOTES: All data, including image files and landmark information, are publicly available at Zenodo (https://zenodo.org/records/13738577). Instructions for using our data can be found on our GitHub page at https://github.com/deshanyang/Liver-DIR-QA. POTENTIAL APPLICATIONS: The landmark dataset generated in this work is the first collection of large-scale liver CT DIR landmarks prepared on real patient images. This dataset can provide researchers with a dense set of ground truth benchmarks for the quantitative evaluation of DIR algorithms within the liver.

A Comprehensive Review of the Recent Advancements in Imaging Segmentation and Registration Techniques for Glioblastoma and Focusing on the Utilization of Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) Scans.

The most common primary malignant brain tumor is glioblastoma. Glioblastoma Multiforme (GBM) diagnosis is difficult. However, image segmentation and registration methods may simplify and automate Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) scan analysis. Medical practitioners and researchers can better identify and characterize glioblastoma tumors using this technology. Many segmentation and registration approaches have been proposed recently. Note that these approaches are not fully compiled. This review efficiently and critically evaluates the state-of-the-art segmentation and registration techniques for MRI and CT GBM images, providing researchers, medical professionals, and students with a wealth of knowledge to advance GBM imaging and inform decision-making. GBM's origins and development have been examined, along with medical imaging methods used to diagnose tumors. Image segmentation and registration were examined, showing their importance in this difficult task. Frequently encountered glioblastoma segmentation and registration issues were examined. Based on these theoretical foundations, recent image segmentation and registration advances were critically analyzed. Additionally, evaluation measures for analytical efforts were thoroughly reviewed.

MUsculo-Skeleton-Aware (MUSA) deep learning for anatomically guided head-and-neck CT deformable registration.

Deep-learning-based deformable image registration (DL-DIR) has demonstrated improved accuracy compared to time-consuming non-DL methods across various anatomical sites. However, DL-DIR is still challenging in heterogeneous tissue regions with large deformation. In fact, several state-of-the-art DL-DIR methods fail to capture the large, anatomically plausible deformation when tested on head-and-neck computed tomography (CT) images. These results allude to the possibility that such complex head-and-neck deformation may be beyond the capacity of a single network structure or a homogeneous smoothness regularization. To address the challenge of combined multi-scale musculoskeletal motion and soft tissue deformation in the head-and-neck region, we propose a MUsculo-Skeleton-Aware (MUSA) framework to anatomically guide DL-DIR by leveraging the explicit multiresolution strategy and the inhomogeneous deformation constraints between the bony structures and soft tissue. The proposed method decomposes the complex deformation into a bulk posture change and residual fine deformation. It can accommodate both inter- and intra- subject registration. Our results show that the MUSA framework can consistently improve registration accuracy and, more importantly, the plausibility of deformation for various network architectures. The code will be publicly available at https://github.com/HengjieLiu/DIR-MUSA.

Evaluation of cumulative dose distributions from external beam radiation therapy using CT-to-CBCT deformable image registration (DIR) for cervical cancer patients.

PURPOSE: To investigate dose differences between the planning CT (pCT) and dose calculated on pre-treatment verification CBCTs using DIR and dose summation for cervical cancer patients. METHODS: Cervical cancer patients treated at our institution with 45 Gy EBRT undergo a pCT and 5 CBCTs, once every five fractions of treatment. A free-form intensity-based DIR in MIM was performed between the pCT and each CBCT using the "Merged CBCT" feature to generate an extended FOV-CBCT (mCBCT). DIR-generated bladder and rectum contours were adjusted by a physician, and dice similarity coefficients (DSC) were calculated. After deformation, the investigated doses were (1) recalculated in Eclipse using original plan parameters (ecD), and (2) deformed from planning dose (pD) using the deformation matrix in MIM (mdD). Dose summation was performed to the first week's mCBCT. Dose distributions were compared for the bladder, rectum, and PTV in terms of percent dose difference, dose volume histograms (DVHs), and gamma analysis between the calculated doses. RESULTS: For the 20 patients, the mean DSC was 0.68 ± 0.17 for bladder and 0.79 ± 0.09 for rectum. Most patients were within 5% of pD for D2cc (19/20), Dmax (17/20), and Dmean (16/20). All patients demonstrated a percent difference > 5% for bladder V45 due to variations in bladder volume from the pCT. D90 showed fewer differences with 19/20 patients within 2% of pD. Gamma rates between pD and ecD averaged 94% for bladder and 94% for rectum, while pD and mdD exhibited slightly better performance for bladder (93%) and lower for rectum (85%). CONCLUSION: Using DIR with weekly CBCT images, the MIM deformed dose (mdD) was found to be in close agreement with the Eclipse calculated dose (ecD). The proposed workflow should be used on a case-by-case basis when the weekly CBCT shows marked difference in organs-at-risk from the planning CT.

Unsupervised Multi-parametric MRI Registration Using Neural Optimal Transport.

Precise deformable image registration of multi-parametric MRI sequences is necessary for radiologists in order to identify abnormalities and diagnose diseases, such as prostate cancer and lymphoma. Despite recent advances in unsupervised learning-based registration, volumetric medical image registration that requires considering the variety of data distributions is still challenging. To address the problem of multi-parametric MRI sequence data registration, we propose an unsupervised domain-transported registration method, called OTMorph by employing neural optimal transport that learns an optimal transport plan to map different data distributions. We have designed a novel framework composed of a transport module and a registration module: the former transports data distribution from the moving source domain to the fixed target domain, and the latter takes the transported data and provides the deformed moving volume that is aligned with the fixed volume. Through end-to-end learning, our proposed method can effectively learn deformable registration for the volumes in different distributions. Experimental results with abdominal multi-parametric MRI sequence data show that our method has superior performance over around 67-85% in deforming the MRI volumes compared to the existing learning-based methods. Our method is generic in nature and can be used to register inter-/intra-modality images by mapping the different data distributions in network training.

Super-resolution reconstruction of time-resolved four-dimensional computed tomography (TR-4DCT) with multiple breathing cycles based on TR-4DMRI.

BACKGROUND: Respiratory motion irregularities in lung cancer patients are common and can be severe during multi-fractional (∼20 mins/fraction) radiotherapy. However, the current clinical standard of motion management is to use a single-breath respiratory-correlated four-dimension computed tomography (RC-4DCT or 4DCT) to estimate tumor motion to delineate the internal tumor volume (ITV), covering the trajectory of tumor motion, as a treatment target. PURPOSE: To develop a novel multi-breath time-resolved (TR) 4DCT using the super-resolution reconstruction framework with TR 4D magnetic resonance imaging (TR-4DMRI) as guidance for patient-specific breathing irregularity assessment, overcoming the shortcomings of RC-4DCT, including binning artifacts and single-breath limitations. METHODS: Six lung cancer patients participated in the IRB-approved protocol study to receive multiple T1w MRI scans, besides an RC-4DCT scan on the simulation day, including 80 low-resolution (lowR: 5 × 5 × 5 mm3) free-breathing (FB) 3D cine MRFB images in 40 s (2 Hz) and a high-resolution (highR: 2 × 2 × 2 mm3) 3D breath-hold (BH) MRBH image for each patient. A CT (1 × 1 × 3 mm3) image was selected from 10-bin RC-4DCT with minimal binning artifacts and a close diaphragm match (<1 cm) to the MRBH image. A mutual-information-based Freeform deformable image registration (DIR) was used to register the CT and MRBH via the opposite directions (namely F1: C T Source → MR Target BH ${\mathrm{C}}{{{\mathrm{T}}}_{{\mathrm{Source}}}} \to {\mathrm{MR}}_{{\mathrm{Target}}}^{{\mathrm{BH}}}$ and F2: C T Target ← MR Source BH ${\mathrm{C}}{{{\mathrm{T}}}_{{\mathrm{Target}}}} \leftarrow {\mathrm{MR}}_{{\mathrm{Source}}}^{{\mathrm{BH}}}$ ) to establish CT-MR voxel correspondences. An intensity-based enhanced Demons DIR was then applied for MR Source BH → MR Target FB ${\mathrm{MR}}_{{\mathrm{Source}}}^{{\mathrm{BH}}} \to {\mathrm{MR}}_{{\mathrm{Target}}}^{{\mathrm{FB}}}$ , in which the original MRBH was used in D1: C T Source → ( MR Source BH → MR Target FB ) Target ${\mathrm{C}}{{{\mathrm{T}}}_{{\mathrm{Source}}}} \to {{({\mathrm{MR}}_{{\mathrm{Source}}}^{{\mathrm{BH}}} \to {\mathrm{MR}}_{{\mathrm{Target}}}^{{\mathrm{FB}}})}_{{\mathrm{Target}}}}$ , while the deformed MRBH was used in D2: ( C T Target ← MR Source BH ) Source → MR Target FB ${{( \text{C}{{\text{T}}_{\text{Target}}}\leftarrow \text{MR}_{\text{Source}}^{\text{BH}} )}_{\text{Source}}}\to \text{MR}_{\text{Target}}^{\text{FB}}$ . The deformation vector fields (DVFs) obtained from each DIR were composed to apply to the deformed CT (D1) and original CT (D2) to reconstruct TR-4DCT images. A digital 4D-XCAT phantom at the end of inhalation (EOI) and end of exhalation (EOE) with 2.5 cm diaphragmatic motion and three spherical targets (ϕ = 2, 3, 4 cm) were first tested to reconstruct TR-4DCT. For each of the six patients, TR-4DCT images at the EOI, middle (MID), and EOE were reconstructed with both D1 and D2 approaches. TR-4DCT image quality was evaluated with mean distance-to-agreement (MDA) at the diaphragm compared with MRFB, tumor volume ratio (TVR) referenced to MRBH, and tumor shape difference (DICE index) compared with the selected input CT. Additionally, differences in the tumor center of mass (|∆COMD1-D2|), together with TVR and DICE comparison, was assessed in the D1 and D2 reconstructed TR-4DCT images. RESULTS: In the phantom, TR-4DCT quality is assessed by MDA = 2.0 ± 0.8 mm at the diaphragm, TVR = 0.8 ± 0.0 for all tumors, and DICE = 0.83 ± 0.01, 0.85 ± 0.02, 0.88 ± 0.01 for ϕ = 2, 3, 4 cm tumors, respectively. In six patients, the MDA in diaphragm match is -1.6 ± 3.1 mm (D1) and 1.0 ± 3.9 mm (D2) between the reconstructed TR-4DCT and lowR MRFB among 18 images (3 phases/patient). The tumor similarity is TVR = 1.2 ± 0.2 and DICE = 0.70 ± 0.07 for D1 and TVR = 1.4 ± 0.3 (D2) and DICE = 0.73 ± 0.07 for D2. The tumor position difference is |∆COMD1-D2| = 1.2 ± 0.8 mm between D1 and D2 reconstructions. CONCLUSION: The feasibility of super-resolution reconstruction of multi-breathing-cycle TR-4DCT is demonstrated and image quality at the diaphragm and tumor is assessed in both the 4D-XCAT phantom and six lung cancer patients. The similarity of D1 and D2 reconstruction suggests consistent and reliable DIR results. Clinically, TR-4DCT has the potential for breathing irregularity assessment and dosimetry evaluation in radiotherapy.

Computed tomography and structured light imaging guided orthopedic navigation puncture system: effective reduction of intraoperative image drift and mismatch.

Background: Image-guided surgical navigation systems are widely regarded as the benchmark for computer-assisted surgical robotic platforms, yet a persistent challenge remains in addressing intraoperative image drift and mismatch. It can significantly impact the accuracy and precision of surgical procedures. Therefore, further research and development are necessary to mitigate this issue and enhance the overall performance of these advanced surgical platforms. Objective: The primary objective is to improve the precision of image guided puncture navigation systems by developing a computed tomography (CT) and structured light imaging (SLI) based navigation system. Furthermore, we also aim to quantifying and visualize intraoperative image drift and mismatch in real time and provide feedback to surgeons, ensuring that surgical procedures are executed with accuracy and reliability. Methods: A CT-SLI guided orthopedic navigation puncture system was developed. Polymer bandages are employed to pressurize, plasticize, immobilize and toughen the surface of a specimen for surgical operations. Preoperative CT images of the specimen are acquired, a 3D navigation map is reconstructed and a puncture path planned accordingly. During surgery, an SLI module captures and reconstructs the 3D surfaces of both the specimen and a guiding tube for the puncture needle. The SLI reconstructed 3D surface of the specimen is matched to the CT navigation map via two-step point cloud registrations, while the SLI reconstructed 3D surface of the guiding tube is fitted by a cylindrical model, which is in turn aligned with the planned puncture path. The proposed system has been tested and evaluated using 20 formalin-soaked lower limb cadaver specimens preserved at a local hospital. Results: The proposed method achieved image registration RMS errors of 0.576 ± 0.146 mm and 0.407 ± 0.234 mm between preoperative CT and intraoperative SLI surface models and between preoperative and postoperative CT surface models. In addition, preoperative and postoperative specimen surface and skeletal drifts were 0.033 ± 0.272 mm and 0.235 ± 0.197 mm respectively. Conclusion: The results indicate that the proposed method is effective in reducing intraoperative image drift and mismatch. The system also visualizes intraoperative image drift and mismatch, and provides real time visual feedback to surgeons.

Patient-specific deep learning tracking framework for real-time 2D target localization in MRI-guided radiotherapy.

PURPOSE: We propose a tumor tracking framework for 2D cine MRI based on a pair of deep learning (DL) models relying on patient-specific (PS) training. METHODS AND MATERIALS: The chosen DL models are: 1) an image registration transformer and 2) an auto-segmentation convolutional neural network (CNN). We collected over 1,400,000 cine MRI frames from 219 patients treated on a 0.35 T MRI-linac plus 7,500 frames from additional 35 patients which were manually labelled and subdivided into fine-tuning, validation, and testing sets. The transformer was first trained on the unlabeled data (without segmentations). We then continued training (with segmentations) either on the fine-tuning set or for PS models based on eight randomly selected frames from the first 5 s of each patient's cine MRI. The PS auto-segmentation CNN was trained from scratch with the same eight frames for each patient, without pre-training. Furthermore, we implemented B-spline image registration as a conventional model, as well as different baselines. Output segmentations of all models were compared on the testing set using the Dice similarity coefficient (DSC), the 50% and 95% Hausdorff distance (HD50%/HD95%), and the root-mean-square-error of the target centroid in superior-inferior direction (RMSESI). RESULTS: The PS transformer and CNN significantly outperformed all other models, achieving a median (inter-quartile range) DSC of 0.92 (0.03)/0.90 (0.04), HD50% of 1.0 (0.1)/1.0 (0.4) mm, HD95% of 3.1 (1.9)/3.8 (2.0) mm and RMSESI of 0.7 (0.4)/0.9 (1.0) mm on the testing set. Their inference time was about 36/8 ms per frame and PS fine-tuning required 3 min for labelling and 8/4 min for training. The transformer was better than the CNN in 9/12 patients, the CNN better in 1/12 patients and the two PS models achieved the same performance on the remaining 2/12 testing patients. CONCLUSION: For targets in the thorax, abdomen and pelvis, we found two PS DL models to provide accurate real-time target localization during MRI-guided radiotherapy.

PViT-AIR: Puzzling vision transformer-based affine image registration for multi histopathology and faxitron images of breast tissue.

Breast cancer is a significant global public health concern, with various treatment options available based on tumor characteristics. Pathological examination of excision specimens after surgery provides essential information for treatment decisions. However, the manual selection of representative sections for histological examination is laborious and subjective, leading to potential sampling errors and variability, especially in carcinomas that have been previously treated with chemotherapy. Furthermore, the accurate identification of residual tumors presents significant challenges, emphasizing the need for systematic or assisted methods to address this issue. In order to enable the development of deep-learning algorithms for automated cancer detection on radiology images, it is crucial to perform radiology-pathology registration, which ensures the generation of accurately labeled ground truth data. The alignment of radiology and histopathology images plays a critical role in establishing reliable cancer labels for training deep-learning algorithms on radiology images. However, aligning these images is challenging due to their content and resolution differences, tissue deformation, artifacts, and imprecise correspondence. We present a novel deep learning-based pipeline for the affine registration of faxitron images, the x-ray representations of macrosections of ex-vivo breast tissue, and their corresponding histopathology images of tissue segments. The proposed model combines convolutional neural networks and vision transformers, allowing it to effectively capture both local and global information from the entire tissue macrosection as well as its segments. This integrated approach enables simultaneous registration and stitching of image segments, facilitating segment-to-macrosection registration through a puzzling-based mechanism. To address the limitations of multi-modal ground truth data, we tackle the problem by training the model using synthetic mono-modal data in a weakly supervised manner. The trained model demonstrated successful performance in multi-modal registration, yielding registration results with an average landmark error of 1.51 mm (±2.40), and stitching distance of 1.15 mm (±0.94). The results indicate that the model performs significantly better than existing baselines, including both deep learning-based and iterative models, and it is also approximately 200 times faster than the iterative approach. This work bridges the gap in the current research and clinical workflow and has the potential to improve efficiency and accuracy in breast cancer evaluation and streamline pathology workflow.

OS-PSO: A Modified Ratio of Exponentially Weighted Averages-Based Optical and SAR Image Registration.

Optical and synthetic aperture radar (SAR) images exhibit non-negligible intensity differences due to their unique imaging mechanisms, which makes it difficult for classical SIFT-based algorithms to obtain sufficiently correct correspondences when processing the registration of these two types of images. To tackle this problem, an accurate optical and SAR image registration algorithm based on the SIFT algorithm (OS-PSO) is proposed. First, a modified ratio of exponentially weighted averages (MROEWA) operator is introduced to resolve the sudden dark patches in SAR images, thus generating more consistent gradients between optical and SAR images. Next, we innovatively construct the Harris scale space to replace the traditional difference in the Gaussian (DoG) scale space, identify repeatable key-points by searching for local maxima, and perform localization refinement on the identified key-points to improve their accuracy. Immediately after that, the gradient location orientation histogram (GLOH) method is adopted to construct the feature descriptors. Finally, we propose an enhanced matching method. The transformed relation is obtained in the initial matching stage using the nearest neighbor distance ratio (NNDR) and fast sample consensus (FSC) methods. And the re-matching takes into account the location, scale, and main direction of key-points to increase the number of correctly corresponding points. The proposed OS-PSO algorithm has been implemented on the Gaofen and Sentinel series with excellent results. The superior performance of the designed registration system can also be applied in complex scenarios, including urban, suburban, river, farmland, and lake areas, with more efficiency and accuracy than the state-of-the-art methods based on the WHU-OPT-SAR dataset and the BISTU-OPT-SAR dataset.

A comparative study on automatic treatment planning for online adaptive proton therapy of esophageal cancer: which combination of deformable registration and deep learning planning tools performs the best?

Objective.To demonstrate the feasibility of integrating fully-automated online adaptive proton therapy strategies (OAPT) within a commercially available treatment planning system and underscore what limits their clinical implementation. These strategies leverage existing deformable image registration (DIR) algorithms and state-of-the-art deep learning (DL) networks for organ segmentation and proton dose prediction.Approach.Four OAPT strategies featuring automatic segmentation and robust optimization were evaluated on a cohort of 17 patients, each undergoing a repeat CT scan. (1) DEF-INIT combines deformably registered contours with template-based optimization. (2) DL-INIT, (3) DL-DEF, and (4) DL-DL employ a nnU-Net DL network for organ segmentation and a controlling ROIs-guided DIR algorithm for internal clinical target volume (iCTV) segmentation. DL-INIT uses this segmentation alongside template-based optimization, DL-DEF integrates it with a dose-mimicking (DM) step using a reference deformed dose, and DL-DL merges it with DM on a reference DL-predicted dose. All strategies were evaluated on manual contours and contours used for optimization and compared with manually adapted plans. Key dose volume metrics like iCTV D98% are reported.Main results.iCTV D98% was comparable in manually adapted plans and for all strategies in nominal cases but dropped to 20 Gy in worst-case scenarios for a few patients per strategy, highlighting the need to correct segmentation errors in the target volume. Evaluations on optimization contours showed minimal relative error, with some outliers, particularly in template-based strategies (DEF-INIT and DL-INIT). DL-DEF achieves a good trade-off between speed and dosimetric quality, showing a passing rate (iCTV D98% > 94%) of 90% when evaluated against 2, 4 and 5 mm setup error and of 88% when evaluated against 7 mm setup error. While template-based methods are more rigid, DL-DEF and DL-DL have potential for further enhancements with proper DM algorithm tuning.Significance.Among investigated strategies, DL-DEF and DL-DL demonstrated promising within 10 min OAPT implementation results and significant potential for improvements.

Tools and recommendations for commissioning and quality assurance of deformable image registration in radiotherapy.

Multiple tools are available for commissioning and quality assurance of deformable image registration (DIR), each with their own advantages and disadvantages in the context of radiotherapy. The selection of appropriate tools should depend on the DIR application with its corresponding available input, desired output, and time requirement. Discussions were hosted by the ESTRO Physics Workshop 2021 on Commissioning and Quality Assurance for DIR in Radiotherapy. A consensus was reached on what requirements are needed for commissioning and quality assurance for different applications, and what combination of tools is associated with this. For commissioning, we recommend the target registration error of manually annotated anatomical landmarks or the distance-to-agreement of manually delineated contours to evaluate alignment. These should be supplemented by the distance to discordance and/or biomechanical criteria to evaluate consistency and plausibility. Digital phantoms can be useful to evaluate DIR for dose accumulation but are currently only available for a limited range of anatomies, image modalities and types of deformations. For quality assurance of DIR for contour propagation, we recommend at least a visual inspection of the registered image and contour. For quality assurance of DIR for warping quantitative information such as dose, Hounsfield units or positron emission tomography-data, we recommend visual inspection of the registered image together with image similarity to evaluate alignment, supplemented by an inspection of the Jacobian determinant or bending energy to evaluate plausibility, and by the dose (gradient) to evaluate relevance. We acknowledge that some of these metrics are still missing in currently available commercial solutions.

SparseMorph: A weakly-supervised lightweight sparse transformer for mono- and multi-modal deformable image registration.

PURPOSE: Deformable image registration (DIR) is crucial for improving the precision of clinical diagnosis. Recent Transformer-based DIR methods have shown promising performance by capturing long-range dependencies. Nevertheless, these methods still grapple with high computational complexity. This work aims to enhance the performance of DIR in both computational efficiency and registration accuracy. METHODS: We proposed a weakly-supervised lightweight Transformer model, named SparseMorph. To reduce computational complexity without compromising the representative feature capture ability, we designed a sparse multi-head self-attention (SMHA) mechanism. To accumulate representative features while preserving high computational efficiency, we constructed a multi-branch multi-layer perception (MMLP) module. Additionally, we developed an anatomically-constrained weakly-supervised strategy to guide the alignment of regions-of-interest in mono- and multi-modal images. RESULTS: We assessed SparseMorph in terms of registration accuracy and computational complexity. Within the mono-modal brain datasets IXI and OASIS, our SparseMorph outperforms the state-of-the-art method TransMatch with improvements of 3.2 % and 2.9 % in DSC scores for MRI-to-CT registration tasks, respectively. Moreover, in the multi-modal cardiac dataset MMWHS, our SparseMorph shows DSC score improvements of 9.7 % and 11.4 % compared to TransMatch in MRI-to-CT and CT-to-MRI registration tasks, respectively. Notably, SparseMorph attains these performance advantages while utilizing 33.33 % of the parameters of TransMatch. CONCLUSIONS: The proposed weakly-supervised deformable image registration model, SparseMorph, demonstrates efficiency in both mono- and multi-modal registration tasks, exhibiting superior performance compared to state-of-the-art algorithms, and establishing an effective DIR method for clinical applications.

Uncertainty Estimation for Dual View X-ray Mammographic Image Registration Using Deep Ensembles.

Techniques are developed for generating uncertainty estimates for convolutional neural network (CNN)-based methods for registering the locations of lesions between the craniocaudal (CC) and mediolateral oblique (MLO) mammographic X-ray image views. Multi-view lesion correspondence is an important task that clinicians perform for characterizing lesions during routine mammographic exams. Automated registration tools can aid in this task, yet if the tools also provide confidence estimates, they can be of greater value to clinicians, especially in cases involving dense tissue where lesions may be difficult to see. A set of deep ensemble-based techniques, which leverage a negative log-likelihood (NLL)-based cost function, are implemented for estimating uncertainties. The ensemble architectures involve significant modifications to an existing CNN dual-view lesion registration algorithm. Three architectural designs are evaluated, and different ensemble sizes are compared using various performance metrics. The techniques are tested on synthetic X-ray data, real 2D X-ray data, and slices from real 3D X-ray data. The ensembles generate covariance-based uncertainty ellipses that are correlated with registration accuracy, such that the ellipse sizes can give a clinician an indication of confidence in the mapping between the CC and MLO views. The results also show that the ellipse sizes can aid in improving computer-aided detection (CAD) results by matching CC/MLO lesion detects and reducing false alarms from both views, adding to clinical utility. The uncertainty estimation techniques show promise as a means for aiding clinicians in confidently establishing multi-view lesion correspondence, thereby improving diagnostic capability.

A Hybrid Approach for Image Acquisition Methods Based on Feature-Based Image Registration.

This paper presents a novel hybrid approach to feature detection designed specifically for enhancing Feature-Based Image Registration (FBIR). Through an extensive evaluation involving state-of-the-art feature detectors such as BRISK, FAST, ORB, Harris, MinEigen, and MSER, the proposed hybrid detector demonstrates superior performance in terms of keypoint detection accuracy and computational efficiency. Three image acquisition methods (i.e., rotation, scene-to-model, and scaling transformations) are considered in the comparison. Applied across a diverse set of remote-sensing images, the proposed hybrid approach has shown marked improvements in match points and match rates, proving its effectiveness in handling varied and complex imaging conditions typical in satellite and aerial imagery. The experimental results have consistently indicated that the hybrid detector outperforms conventional methods, establishing it as a valuable tool for advanced image registration tasks.

Comparison of image registration methods in patients with non-melanoma skin cancer treated with superficial brachytherapy.

The accumulated dose from sequential treatments of metachronous non-melanoma skin cancer can be assessed using image registration, although guidelines for selecting the appropriate algorithm are lacking. This study shows the impact of rigid (RIR), deformable (DIR) and deformable structure-based (SDIR) algorithms on the skin dose. DIR increased: the maximum dose (39.2 Gy vs 9.4 Gy), the dose to 0.1 cm3 (16.4 Gy vs 7.8 Gy) and the dose to 2 cm3 (7.6 Gy vs 5.7 Gy). RIR only affected the maximum dose, which increased to 17.0 Gy. SDIR correctly translated the dose maps, as none of the parameters changed significantly.

Multilevel network for large deformation image registration based on feature consistency and flow normalization.

BACKGROUND: Deformable image registration is an essential technique of medical image analysis, which plays important roles in several clinical applications. Existing deep learning-based registration methods have already achieved promising performance for the registrations with small deformations, while it is still challenging to deal with the large deformation registration due to the limits of the image intensity-similarity-based objective function. PURPOSE: To achieve the image registration with large-scale deformations, we proposed a multilevel network architecture FCNet to gradually refine the registration results based on semantic feature consistency constraint and flow normalization (FN) strategy. METHODS: At each level of FCNet, the architecture is mainly composed to a FeaExtractor, a FN module, and a spatial transformation module. FeaExtractor consists of three parallel streams which are used to extract the individual features of fixed and moving images, as well as their joint features, respectively. Using these features, the initial deformation field is estimated, which passes through a FN module to refine the deformation field based on the difference map of deformation filed between two adjacent levels. This allows the FCNet to progressively improve the registration performance. Finally, a spatial transformation module is used to get the warped image based on the deformation field. Moreover, in addition to the image intensity-similarity-based objective function, a semantic-feature consistency constraint is also introduced, which can further promote the alignments by imposing the similarity between the fixed and warped image features. To validate the effectiveness of the proposed method, we compared our method with the state-of-the-art methods on three different datasets. In EMPIRE10 dataset, 20, 3, and 7 fixed and moving 3D computer tomography (CT) image pairs were used for training, validation, and testing respectively; in IXI dataset, atlas to individual image registration task was performed, with 3D MR images of 408, 58, and 115 individuals were used for training, validation, and testing respectively; in the in-house dataset, patient to atlas registration task was implemented, with the 3D MR images of 94, 3, and 15 individuals being training, validation, and testing sets, respectively. RESULTS: The qualitative and quantitative comparison results demonstrated that the proposed method is beneficial for handling large deformation image registration problems, with the DSC and ASSD improved by at least 1.0% and 25.9% on EMPIRE10 dataset. The ablation experiments also verified the effectiveness of the proposed feature combination strategy, feature consistency constraint, and FN module. CONCLUSIONS: Our proposed FCNet enables multiscale registration from coarse to fine, surpassing existing SOTA registration methods and effectively handling long-range spatial relationships.

Atlas-based assessment of hypomyelination: Quantitative MRI in Pelizaeus-Merzbacher disease.

Pelizaeus-Merzbacher disease (PMD) is a rare childhood hypomyelinating leukodystrophy. Quantification of the pronounced myelin deficit and delineation of subtle myelination processes are of high clinical interest. Quantitative magnetic resonance imaging (qMRI) techniques can provide in vivo insights into myelination status, its spatial distribution, and dynamics during brain maturation. They may serve as potential biomarkers to assess the efficacy of myelin-modulating therapies. However, registration techniques for image quantification and statistical comparison of affected pediatric brains, especially those of low or deviant image tissue contrast, with healthy controls are not yet established. This study aimed first to develop and compare postprocessing pipelines for atlas-based quantification of qMRI data in pediatric patients with PMD and evaluate their registration accuracy. Second, to apply an optimized pipeline to investigate spatial myelin deficiency using myelin water imaging (MWI) data from patients with PMD and healthy controls. This retrospective single-center study included five patients with PMD (mean age, 6 years ± 3.8) who underwent conventional brain MRI and diffusion tensor imaging (DTI), with MWI data available for a subset of patients. Three methods of registering PMD images to a pediatric template were investigated. These were based on (a) T1-weighted (T1w) images, (b) fractional anisotropy (FA) maps, and (c) a combination of T1w, T2-weighted, and FA images in a multimodal approach. Registration accuracy was determined by visual inspection and calculated using the structural similarity index method (SSIM). SSIM values for the registration approaches were compared using a t test. Myelin water fraction (MWF) was quantified from MWI data as an assessment of relative myelination. Mean MWF was obtained from two PMDs (mean age, 3.1 years ± 0.3) within four major white matter (WM) pathways of a pediatric atlas and compared to seven healthy controls (mean age, 3 years ± 0.2) using a Mann-Whitney U test. Our results show that visual registration accuracy estimation and computed SSIM were highest for FA-based registration, followed by multimodal, and T1w-based registration (SSIMFA = 0.67 ± 0.04 vs. SSIMmultimodal = 0.60 ± 0.03 vs. SSIMT1 = 0.40 ± 0.14). Mean MWF of patients with PMD within the WM pathways was significantly lower than in healthy controls MWFPMD = 0.0267 ± 0.021 vs. MWFcontrols = 0.1299 ± 0.039. Specifically, MWF was measurable in brain structures known to be myelinated at birth (brainstem) or postnatally (projection fibers) but was scarcely detectable in other brain regions (commissural and association fibers). Taken together, our results indicate that registration accuracy was highest with an FA-based registration pipeline, providing an alternative to conventional T1w-based registration approaches in the case of hypomyelinating leukodystrophies missing normative intrinsic tissue contrasts. The applied atlas-based analysis of MWF data revealed that the extent of spatial myelin deficiency in patients with PMD was most pronounced in commissural and association and to a lesser degree in brainstem and projection pathways.

Multiscale unsupervised network for deformable image registration.

BACKGROUND: Deformable image registration (DIR) plays an important part in many clinical tasks, and deep learning has made significant progress in DIR over the past few years. OBJECTIVE: To propose a fast multiscale unsupervised deformable image registration (referred to as FMIRNet) method for monomodal image registration. METHODS: We designed a multiscale fusion module to estimate the large displacement field by combining and refining the deformation fields of three scales. The spatial attention mechanism was employed in our fusion module to weight the displacement field pixel by pixel. Except mean square error (MSE), we additionally added structural similarity (ssim) measure during the training phase to enhance the structural consistency between the deformed images and the fixed images. RESULTS: Our registration method was evaluated on EchoNet, CHAOS and SLIVER, and had indeed performance improvement in terms of SSIM, NCC and NMI scores. Furthermore, we integrated the FMIRNet into the segmentation network (FCN, UNet) to boost the segmentation task on a dataset with few manual annotations in our joint leaning frameworks. The experimental results indicated that the joint segmentation methods had performance improvement in terms of Dice, HD and ASSD scores. CONCLUSIONS: Our proposed FMIRNet is effective for large deformation estimation, and its registration capability is generalizable and robust in joint registration and segmentation frameworks to generate reliable labels for training segmentation tasks.