Association of iron rim lesions with brain and cervical cord volume in relapsing multiple sclerosis.

OBJECTIVES: In multiple sclerosis (MS), iron rim lesions (IRLs) are indicators of chronic low-grade inflammation and ongoing tissue destruction. The aim of this study was to assess the relationship of IRLs with clinical measures and magnetic resonance imaging (MRI) markers, in particular brain and cervical cord volume. METHODS: Clinical and MRI parameters from 102 relapsing MS patients (no relapses for at least 6 months, no contrast-enhancing lesions) were included; follow-up data obtained after 12 months was available in 49 patients. IRLs were identified on susceptibility-weighted images (SWIs). In addition to standard brain and spinal cord MRI parameters, normalised cross-sectional area (nCSA) of the upper cervical cord was calculated. RESULTS: Thirty-eight patients had at least one IRL on SWI MRI. At baseline, patients with IRLs had higher EDSS scores, higher lesion loads (brain and spinal cord), and lower cortical grey matter volumes and a lower nCSA. At follow-up, brain atrophy rates were higher in patients with IRLs. IRLs correlated spatially with T1-hypointense lesions. CONCLUSIONS: Relapsing MS patients with IRLs showed more aggressive MRI disease characteristics in both the cross-sectional and longitudinal analyses. KEY POINTS: • Multiple sclerosis patients with iron rim lesions had higher EDSS scores, higher brain and spinal cord lesion loads, lower cortical grey matter volumes, and a lower normalised cross-sectional area of the upper cervical spinal cord. • Iron rim lesions are a new lesion descriptor obtained from susceptibility-weighted MRI. Our data suggests that further exploration of this lesion characteristic in regard to a poorer prognosis in multiple sclerosis patients is warranted.

Integration of 3D multimodal imaging data of a head and neck cancer and advanced feature recognition.

In the last years, matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) became an imaging technique which has the potential to characterize complex tumor tissue. The combination with other modalities and with standard histology techniques was achieved by the use of image registration methods and enhances analysis possibilities. We analyzed an oral squamous cell carcinoma with up to 162 consecutive sections with MALDI MSI, hematoxylin and eosin (H&E) staining and immunohistochemistry (IHC) against CD31. Spatial segmentation maps of the MALDI MSI data were generated by similarity-based clustering of spectra. Next, the maps were overlaid with the H&E microscopy images and the results were interpreted by an experienced pathologist. Image registration was used to fuse both modalities and to build a three-dimensional (3D) model. To visualize structures below resolution of MALDI MSI, IHC was carried out for CD31 and results were embedded additionally. The integration of 3D MALDI MSI data with H&E and IHC images allows a correlation between histological and molecular information leading to a better understanding of the functional heterogeneity of tumors. This article is part of a Special Issue entitled: MALDI Imaging, edited by Dr. Corinna Henkel and Prof. Peter Hoffmann.

2D and 3D MALDI-imaging: conceptual strategies for visualization and data mining.

3D imaging has a significant impact on many challenges in life sciences, because biology is a 3-dimensional phenomenon. Current 3D imaging-technologies (various types MRI, PET, SPECT) are labeled, i.e. they trace the localization of a specific compound in the body. In contrast, 3D MALDI mass spectrometry-imaging (MALDI-MSI) is a label-free method imaging the spatial distribution of molecular compounds. It complements 3D imaging labeled methods, immunohistochemistry, and genetics-based methods. However, 3D MALDI-MSI cannot tap its full potential due to the lack of statistical methods for analysis and interpretation of large and complex 3D datasets. To overcome this, we established a complete and robust 3D MALDI-MSI pipeline combined with efficient computational data analysis methods for 3D edge preserving image denoising, 3D spatial segmentation as well as finding colocalized m/z values, which will be reviewed here in detail. Furthermore, we explain, why the integration and correlation of the MALDI imaging data with other imaging modalities allows to enhance the interpretation of the molecular data and provides visualization of molecular patterns that may otherwise not be apparent. Therefore, a 3D data acquisition workflow is described generating a set of 3 different dimensional images representing the same anatomies. First, an in-vitro MRI measurement is performed which results in a three-dimensional image modality representing the 3D structure of the measured object. After sectioning the 3D object into N consecutive slices, all N slices are scanned using an optical digital scanner, enabling for performing the MS measurements. Scanning the individual sections results into low-resolution images, which define the base coordinate system for the whole pipeline. The scanned images conclude the information from the spatial (MRI) and the mass spectrometric (MALDI-MSI) dimension and are used for the spatial three-dimensional reconstruction of the object performed by image registration techniques. Different strategies for automatic serial image registration applied to MS datasets are outlined in detail. The third image modality is histology driven, i.e. a digital scan of the histological stained slices in high-resolution. After fusion of reconstructed scan images and MRI the slice-related coordinates of the mass spectra can be propagated into 3D-space. After image registration of scan images and histological stained images, the anatomical information from histology is fused with the mass spectra from MALDI-MSI. As a result of the described pipeline we have a set of 3 dimensional images representing the same anatomies, i.e. the reconstructed slice scans, the spectral images as well as corresponding clustering results, and the acquired MRI. Great emphasis is put on the fact that the co-registered MRI providing anatomical details improves the interpretation of 3D MALDI images. The ability to relate mass spectrometry derived molecular information with in vivo and in vitro imaging has potentially important implications. This article is part of a Special Issue entitled: Computational Proteomics in the Post-Identification Era. Guest Editors: Martin Eisenacher and Christian Stephan.

Generative adversarial network-based reconstruction of healthy anatomy for anomaly detection in brain CT scans.

Purpose: To help radiologists examine the growing number of computed tomography (CT) scans, automatic anomaly detection is an ongoing focus of medical imaging research. Radiologists must analyze a CT scan by searching for any deviation from normal healthy anatomy. We propose an approach to detecting abnormalities in axial 2D CT slice images of the brain. Although much research has been done on detecting abnormalities in magnetic resonance images of the brain, there is little work on CT scans, where abnormalities are more difficult to detect due to the low image contrast that must be represented by the model used. Approach: We use a generative adversarial network (GAN) to learn normal brain anatomy in the first step and compare two approaches to image reconstruction: training an encoder in the second step and using iterative optimization during inference. Then, we analyze the differences from the original scan to detect and localize anomalies in the brain. Results: Our approach can reconstruct healthy anatomy with good image contrast for brain CT scans. We obtain median Dice scores of 0.71 on our hemorrhage test data and 0.43 on our test set with additional tumor images from publicly available data sources. We also compare our models to a state-of-the-art autoencoder and a diffusion model and obtain qualitatively more accurate reconstructions. Conclusions: Without defining anomalies during training, a GAN-based network was used to learn healthy anatomy for brain CT scans. Notably, our approach is not limited to the localization of hemorrhages and tumors and could thus be used to detect structural anatomical changes and other lesions.

Learning three-dimensional aortic root assessment based on sparse annotations.

Purpose: Analyzing the anatomy of the aorta and left ventricular outflow tract (LVOT) is crucial for risk assessment and planning of transcatheter aortic valve implantation (TAVI). A comprehensive analysis of the aortic root and LVOT requires the extraction of the patient-individual anatomy via segmentation. Deep learning has shown good performance on various segmentation tasks. If this is formulated as a supervised problem, large amounts of annotated data are required for training. Therefore, minimizing the annotation complexity is desirable. Approach: We propose two-dimensional (2D) cross-sectional annotation and point cloud-based surface reconstruction to train a fully automatic 3D segmentation network for the aortic root and the LVOT. Our sparse annotation scheme enables easy and fast training data generation for tubular structures such as the aortic root. From the segmentation results, we derive clinically relevant parameters for TAVI planning. Results: The proposed 2D cross-sectional annotation results in high inter-observer agreement [Dice similarity coefficient (DSC): 0.94]. The segmentation model achieves a DSC of 0.90 and an average surface distance of 0.96 mm. Our approach achieves an aortic annulus maximum diameter difference between prediction and annotation of 0.45 mm (inter-observer variance: 0.25 mm). Conclusions: The presented approach facilitates reproducible annotations. The annotations allow for training accurate segmentation models of the aortic root and LVOT. The segmentation results facilitate reproducible and quantifiable measurements for TAVI planning.

The ACROBAT 2022 challenge: Automatic registration of breast cancer tissue.

The alignment of tissue between histopathological whole-slide-images (WSI) is crucial for research and clinical applications. Advances in computing, deep learning, and availability of large WSI datasets have revolutionised WSI analysis. Therefore, the current state-of-the-art in WSI registration is unclear. To address this, we conducted the ACROBAT challenge, based on the largest WSI registration dataset to date, including 4,212 WSIs from 1,152 breast cancer patients. The challenge objective was to align WSIs of tissue that was stained with routine diagnostic immunohistochemistry to its H&E-stained counterpart. We compare the performance of eight WSI registration algorithms, including an investigation of the impact of different WSI properties and clinical covariates. We find that conceptually distinct WSI registration methods can lead to highly accurate registration performances and identify covariates that impact performances across methods. These results provide a comparison of the performance of current WSI registration methods and guide researchers in selecting and developing methods.

Comparison of consecutive and restained sections for image registration in histopathology.

Significance: Although the registration of restained sections allows nucleus-level alignment that enables a direct analysis of interacting biomarkers, consecutive sections only allow the transfer of region-level annotations. The latter can be achieved at low computational cost using coarser image resolutions. Purpose: In digital histopathology, virtual multistaining is important for diagnosis and biomarker research. Additionally, it provides accurate ground truth for various deep-learning tasks. Virtual multistaining can be obtained using different stains for consecutive sections or by restaining the same section. Both approaches require image registration to compensate for tissue deformations, but little attention has been devoted to comparing their accuracy. Approach: We compared affine and deformable variational image registration of consecutive and restained sections and analyzed the effect of the image resolution that influences accuracy and required computational resources. The registration was applied to the automatic nonrigid histological image registration (ANHIR) challenge data (230 consecutive slide pairs) and the hyperparameters were determined. Then without changing the parameters, the registration was applied to a newly published hybrid dataset of restained and consecutive sections (HyReCo, 86 slide pairs, 5404 landmarks). Results: We obtain a median landmark error after registration of 6.5 µm (HyReCo) and 24.1 µm (ANHIR) between consecutive sections. Between restained sections, the median registration error is 2.2 and 0.9 µm in the two subsets of the HyReCo dataset. We observe that deformable registration leads to lower landmark errors than affine registration in both cases (p<0.001), though the effect is smaller in restained sections. Conclusion: Deformable registration of consecutive and restained sections is a valuable tool for the joint analysis of different stains.

MR-CT multi-atlas registration guided by fully automated brain structure segmentation with CNNs.

PURPOSE: Computed tomography (CT) is widely used to identify anomalies in brain tissues because their localization is important for diagnosis and therapy planning. Due to the insufficient soft tissue contrast of CT, the division of the brain into anatomical meaningful regions is challenging and is commonly done with magnetic resonance imaging (MRI). METHODS: We propose a multi-atlas registration approach to propagate anatomical information from a standard MRI brain atlas to CT scans. This translation will enable a detailed automated reporting of brain CT exams. We utilize masks of the lateral ventricles and the brain volume of CT images as adjuvant input to guide the registration process. Besides using manual annotations to test the registration in a first step, we then verify that convolutional neural networks (CNNs) are a reliable solution for automatically segmenting structures to enhance the registration process. RESULTS: The registration method obtains mean Dice values of 0.92 and 0.99 in brain ventricles and parenchyma on 22 healthy test cases when using manually segmented structures as guidance. When guiding with automatically segmented structures, the mean Dice values are 0.87 and 0.98, respectively. CONCLUSION: Our registration approach is a fully automated solution to register MRI atlas images to CT scans and thus obtain detailed anatomical information. The proposed CNN segmentation method can be used to obtain masks of ventricles and brain volume which guide the registration.

Deep learning-based segmentation of brain parenchyma and ventricular system in CT scans in the presence of anomalies.

Introduction: The automatic segmentation of brain parenchyma and cerebrospinal fluid-filled spaces such as the ventricular system is the first step for quantitative and qualitative analysis of brain CT data. For clinical practice and especially for diagnostics, it is crucial that such a method is robust to anatomical variability and pathological changes such as (hemorrhagic or neoplastic) lesions and chronic defects. This study investigates the increase in overall robustness of a deep learning algorithm that is gained by adding hemorrhage training data to an otherwise normal training cohort. Methods: A 2D U-Net is trained on subjects with normal appearing brain anatomy. In a second experiment the training data includes additional subjects with brain hemorrhage on image data of the RSNA Brain CT Hemorrhage Challenge with custom reference segmentations. The resulting networks are evaluated on normal and hemorrhage test casesseparately, and on an independent test set of patients with brain tumors of the publicly available GLIS-RT dataset. Results: Adding data with hemorrhage to the training set significantly improves the segmentation performance over an algorithm trained exclusively on normally appearing data, not only in the hemorrhage test set but also in the tumor test set. The performance on normally appearing data is stable. Overall, the improved algorithm achieves median Dice scores of 0.98 (parenchyma), 0.91 (left ventricle), 0.90 (right ventricle), 0.81 (third ventricle), and 0.80 (fourth ventricle) on the hemorrhage test set. On the tumor test set, the median Dice scores are 0.96 (parenchyma), 0.90 (left ventricle), 0.90 (right ventricle), 0.75 (third ventricle), and 0.73 (fourth ventricle). Conclusion: Training on an extended data set that includes pathologies is crucial and significantly increases the overall robustness of a segmentation algorithm for brain parenchyma and ventricular system in CT data, also for anomalies completely unseen during training. Extension of the training set to include other diseases may further improve the generalizability of the algorithm.

Assessing Osteolytic Lesion Size on Sequential CT Scans Is a Reliable Study Endpoint for Bone Remineralization in Newly Diagnosed Multiple Myeloma.

Multiple myeloma (MM) frequently induces persisting osteolytic manifestations despite hematologic treatment response. This study aimed to establish a biometrically valid study endpoint for bone remineralization through quantitative and qualitative analyses in sequential CT scans. Twenty patients (seven women, 58 ± 8 years) with newly diagnosed MM received standardized induction therapy comprising the anti-SLAMF7 antibody elotuzumab, carfilzomib, lenalidomide, and dexamethasone (E-KRd). All patients underwent whole-body low-dose CT scans before and after six cycles of E-KRd. Two radiologists independently recorded osteolytic lesion sizes, as well as the presence of cortical destruction, pathologic fractures, rim and trabecular sclerosis. Bland-Altman analyses and Krippendorff's α were employed to assess inter-reader reliability, which was high for lesion size measurement (standard error 1.2 mm) and all qualitative criteria assessed (α ≥ 0.74). After six cycles of E-KRd induction, osteolytic lesion size decreased by 22% (p < 0.001). While lesion size response did not correlate with the initial lesion size at baseline imaging (Pearson's r = 0.144), logistic regression analysis revealed that the majority of responding osteolyses exhibited trabecular sclerosis (p < 0.001). The sum of osteolytic lesion sizes on sequential CT scans defines a reliable study endpoint to characterize bone remineralization. Patient level response is strongly associated with the presence of trabecular sclerosis.

Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https://learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

Exploring three-dimensional matrix-assisted laser desorption/ionization imaging mass spectrometry data: three-dimensional spatial segmentation of mouse kidney.

Three-dimensional (3D) imaging has a significant impact on many challenges of life sciences. Three-dimensional matrix-assisted laser desorption/ionization imaging mass spectrometry (MALDI-IMS) is an emerging label-free bioanalytical technique capturing the spatial distribution of hundreds of molecular compounds in 3D by providing a MALDI mass spectrum for each spatial point of a 3D sample. Currently, 3D MALDI-IMS cannot tap its full potential due to the lack efficient computational methods for constructing, processing, and visualizing large and complex 3D MALDI-IMS data. We present a new pipeline of efficient computational methods, which enables analysis and interpretation of a 3D MALDI-IMS data set. Construction of a MALDI-IMS data set was done according to the state-of-the-art protocols and involved sample preparation, spectra acquisition, spectra preprocessing, and registration of serial sections. For analysis and interpretation of 3D MALDI-IMS data, we applied the spatial segmentation approach which is well-accepted in analysis of two-dimensional (2D) MALDI-IMS data. In line with 2D data analysis, we used edge-preserving 3D image denoising prior to segmentation to reduce strong and chaotic spectrum-to-spectrum variation. For segmentation, we used an efficient clustering method, called bisecting k-means, which is optimized for hierarchical clustering of a large 3D MALDI-IMS data set. Using the proposed pipeline, we analyzed a central part of a mouse kidney using 33 serial sections of 3.5 µm thickness after the PAXgene tissue fixation and paraffin embedding. For each serial section, a 2D MALDI-IMS data set was acquired following the standard protocols with the high spatial resolution of 50 µm. Altogether, 512 495 mass spectra were acquired that corresponds to approximately 50 gigabytes of data. After registration of serial sections into a 3D data set, our computational pipeline allowed us to reveal the 3D kidney anatomical structure based on mass spectrometry data only. Finally, automated analysis discovered molecular masses colocalized with major anatomical regions. In the same way, the proposed pipeline can be used for analysis and interpretation of any 3D MALDI-IMS data set in particular of pathological cases.

In-vivo assessment of meniscal movement in the knee joint during internal and external rotation under load.

PURPOSE: The menisci transmit load between femur and tibia and thus play a crucial role in the functionality of the knee joint. Knee joint movements have a major impact on the position of the menisci. However, these meniscus movements have not yet been assessed in a validated setting. The objective of this study is to evaluate the meniscal movements in MRI with prospective motion correction based on optical tracking under loading via internal and external tibial torques.  METHODS: Thirty-one healthy volunteers were recruited for this study. MRI scans were performed in internal and external rotation induced by a torque of 5 Nm, using a 3 T MRI. A validated software used the generated images to calculate the absolute meniscus movements as the sum of all vectors. Differences between subgroups were analyzed by using a Wilcoxon signed-rank test.  RESULTS: The MM shows an average movement of 1.79 mm in anterior-lateral direction under internal rotation and 6.01 mm in posterior-lateral direction under external rotation, whereas the LM moves an average of 4.55 mm in posterior-medial direction under internal rotation and 3.58 mm in anterior-medial direction under external rotation. When comparing the overall meniscus movements between internal and external rotation, statistically significant differences were found for total vector length and the direction of meniscus movements for medial and lateral meniscus. The comparison between medial and lateral meniscus movements also showed statistically significant differences in all categories for internal and external rotation. CONCLUSIONS: Overall, the MM and LM movements in internal and external rotation differ significantly in extent and direction, although MM and LM movements in opposite directions during internal and external rotation can be observed. In internal rotation, most meniscus movements were found in the IHLM. In external rotation, the IHMM showed the greatest mobility. Segment analysis of internal vs. external rotation showed less difference in LM movements than MM. LEVEL OF EVIDENCE: Level II.

Quantitative evaluation of the influence of multiple MRI sequences and of pathological tissues on the registration of longitudinal data acquired during brain tumor treatment.

Registration methods facilitate the comparison of multiparametric magnetic resonance images acquired at different stages of brain tumor treatments. Image-based registration solutions are influenced by the sequences chosen to compute the distance measure, and the lack of image correspondences due to the resection cavities and pathological tissues. Nonetheless, an evaluation of the impact of these input parameters on the registration of longitudinal data is still missing. This work evaluates the influence of multiple sequences, namely T1-weighted (T1), T2-weighted (T2), contrast enhanced T1-weighted (T1-CE), and T2 Fluid Attenuated Inversion Recovery (FLAIR), and the exclusion of the pathological tissues on the non-rigid registration of pre- and post-operative images. We here investigate two types of registration methods, an iterative approach and a convolutional neural network solution based on a 3D U-Net. We employ two test sets to compute the mean target registration error (mTRE) based on corresponding landmarks. In the first set, markers are positioned exclusively in the surroundings of the pathology. The methods employing T1-CE achieves the lowest mTREs, with a improvement up to 0.8 mm for the iterative solution. The results are higher than the baseline when using the FLAIR sequence. When excluding the pathology, lower mTREs are observable for most of the methods. In the second test set, corresponding landmarks are located in the entire brain volumes. Both solutions employing T1-CE obtain the lowest mTREs, with a decrease up to 1.16 mm for the iterative method, whereas the results worsen using the FLAIR. When excluding the pathology, an improvement is observable for the CNN method using T1-CE. Both approaches utilizing the T1-CE sequence obtain the best mTREs, whereas the FLAIR is the least informative to guide the registration process. Besides, the exclusion of pathology from the distance measure computation improves the registration of the brain tissues surrounding the tumor. Thus, this work provides the first numerical evaluation of the influence of these parameters on the registration of longitudinal magnetic resonance images, and it can be helpful for developing future algorithms.

CNN-based lung CT registration with multiple anatomical constraints.

Deep-learning-based registration methods emerged as a fast alternative to conventional registration methods. However, these methods often still cannot achieve the same performance as conventional registration methods because they are either limited to small deformation or they fail to handle a superposition of large and small deformations without producing implausible deformation fields with foldings inside. In this paper, we identify important strategies of conventional registration methods for lung registration and successfully developed the deep-learning counterpart. We employ a Gaussian-pyramid-based multilevel framework that can solve the image registration optimization in a coarse-to-fine fashion. Furthermore, we prevent foldings of the deformation field and restrict the determinant of the Jacobian to physiologically meaningful values by combining a volume change penalty with a curvature regularizer in the loss function. Keypoint correspondences are integrated to focus on the alignment of smaller structures. We perform an extensive evaluation to assess the accuracy, the robustness, the plausibility of the estimated deformation fields, and the transferability of our registration approach. We show that it achieves state-of-the-art results on the COPDGene dataset compared to conventional registration method with much shorter execution time. In our experiments on the DIRLab exhale to inhale lung registration, we demonstrate substantial improvements (TRE below 1.2 mm) over other deep learning methods. Our algorithm is publicly available at https://grand-challenge.org/algorithms/deep-learning-based-ct-lung-registration/.

Instrument localisation for endovascular aneurysm repair: Comparison of two methods based on tracking systems or using imaging.

BACKGROUND: In endovascular aneuysm repair (EVAR) procedures, medical instruments are currently navigated with a two-dimensional imaging based guidance requiring X-rays and contrast agent. METHODS: Novel approaches for obtaining the three-dimensional instrument positions are introduced. Firstly, a method based on fibre optical shape sensing, one electromagnetic sensor and a preoperative computed tomography (CT) scan is described. Secondly, an approach based on image processing using one 2D fluoroscopic image and a preoperative CT scan is introduced. RESULTS: For the tracking based method, average errors from 1.81 to 3.13 mm and maximum errors from 3.21 to 5.46 mm were measured. For the image-based approach, average errors from 3.07 to 6.02 mm and maximum errors from 8.05 to 15.75 mm were measured. CONCLUSION: The tracking based method is promising for usage in EVAR procedures. For the image-based approach are applications in smaller vessels more suitable, since its errors increase with the vessel diameter.

ANHIR: Automatic Non-Rigid Histological Image Registration Challenge.

Automatic Non-rigid Histological Image Registration (ANHIR) challenge was organized to compare the performance of image registration algorithms on several kinds of microscopy histology images in a fair and independent manner. We have assembled 8 datasets, containing 355 images with 18 different stains, resulting in 481 image pairs to be registered. Registration accuracy was evaluated using manually placed landmarks. In total, 256 teams registered for the challenge, 10 submitted the results, and 6 participated in the workshop. Here, we present the results of 7 well-performing methods from the challenge together with 6 well-known existing methods. The best methods used coarse but robust initial alignment, followed by non-rigid registration, used multiresolution, and were carefully tuned for the data at hand. They outperformed off-the-shelf methods, mostly by being more robust. The best methods could successfully register over 98% of all landmarks and their mean landmark registration accuracy (TRE) was 0.44% of the image diagonal. The challenge remains open to submissions and all images are available for download.

Memory-efficient 2.5D convolutional transformer networks for multi-modal deformable registration with weak label supervision applied to whole-heart CT and MRI scans.

PURPOSE : Despite its potential for improvements through supervision, deep learning-based registration approaches are difficult to train for large deformations in 3D scans due to excessive memory requirements. METHODS : We propose a new 2.5D convolutional transformer architecture that enables us to learn a memory-efficient weakly supervised deep learning model for multi-modal image registration. Furthermore, we firstly integrate a volume change control term into the loss function of a deep learning-based registration method to penalize occurring foldings inside the deformation field. RESULTS : Our approach succeeds at learning large deformations across multi-modal images. We evaluate our approach on 100 pair-wise registrations of CT and MRI whole-heart scans and demonstrate considerably higher Dice Scores (of 0.74) compared to a state-of-the-art unsupervised discrete registration framework (deeds with Dice of 0.71). CONCLUSION : Our proposed memory-efficient registration method performs better than state-of-the-art conventional registration methods. By using a volume change control term in the loss function, the number of occurring foldings can be considerably reduced on new registration cases.

Accuracy of registration algorithms in subtraction CT of the lungs: A digital phantom study.

PURPOSE: The purpose of this study was to assess, using an anthropomorphic digital phantom, the accuracy of algorithms in registering precontrast and contrast-enhanced computed tomography (CT) chest images for generation of iodine maps of the pulmonary parenchyma via temporal subtraction. MATERIALS AND METHODS: The XCAT phantom, with enhanced airway and pulmonary vessel structures, was used to simulate precontrast and contrast-enhanced chest images at various inspiration levels and added CT simulation for realistic system noise. Differences in diaphragm position were varied between 0 and 20 mm, with the maximum chosen to exceed the 95th percentile found in a dataset of 100 clinical subtraction CTs. In addition, the influence of whole body movement, degree of iodine enhancement, beam hardening artifacts, presence of nodules and perfusion defects in the pulmonary parenchyma, and variation in noise on the registration were also investigated. Registration was performed using three lung registration algorithms - a commercial (algorithm A) and a prototype (algorithm B) version from Canon Medical Systems and an algorithm from the MEVIS Fraunhofer institute (algorithm C). For each algorithm, we calculated the voxel-by-voxel difference between the true deformation and the algorithm-estimated deformation in the lungs. RESULTS: The median absolute residual error for all three algorithms was smaller than the voxel size (1.0 × 1.0 × 1.0 mm3 ) for up to an 8 mm diaphragm difference, which is the average difference in diaphragm levels found clinically, and increased with increasing difference in diaphragm position. At 20 mm diaphragm displacement, the median absolute residual error after registration was 0.85 mm (interquartile range, 0.51-1.47 mm) for algorithm A, 0.82 mm (0.50-1.40 mm) for algorithm B, and 0.91 mm (0.54-1.52 mm) for algorithm C. The largest errors were seen in the paracardiac regions and close to the diaphragm. The impact of all other evaluated conditions on the residual error varied, resulting in an increase in the median residual error lower than 0.1 mm for all algorithms, except in the case of whole body displacements for algorithm B, and with increased noise for algorithm C. CONCLUSION: Motion correction software can compensate for respiratory and cardiac motion with a median residual error below 1 mm, which was smaller than the voxel size, with small differences among the tested registration algorithms for different conditions. Perfusion defects above 50 mm will be visible with the commercially available subtraction CT software, even in poorly registered areas, where the median residual error in that area was 7.7 mm.

Estimation of Large Motion in Lung CT by Integrating Regularized Keypoint Correspondences into Dense Deformable Registration.

We present a novel algorithm for the registration of pulmonary CT scans. Our method is designed for large respiratory motion by integrating sparse keypoint correspondences into a dense continuous optimization framework. The detection of keypoint correspondences enables robustness against large deformations by jointly optimizing over a large number of potential discrete displacements, whereas the dense continuous registration achieves subvoxel alignment with smooth transformations. Both steps are driven by the same normalized gradient fields data term. We employ curvature regularization and a volume change control mechanism to prevent foldings of the deformation grid and restrict the determinant of the Jacobian to physiologically meaningful values. Keypoint correspondences are integrated into the dense registration by a quadratic penalty with adaptively determined weight. Using a parallel matrix-free derivative calculation scheme, a runtime of about 5 min was realized on a standard PC. The proposed algorithm ranks first in the EMPIRE10 challenge on pulmonary image registration. Moreover, it achieves an average landmark distance of 0.82 mm on the DIR-Lab COPD database, thereby improving upon the state of the art in accuracy by 15%. Our algorithm is the first to reach the inter-observer variability in landmark annotation on this dataset.