VertXNet: an ensemble method for vertebral body segmentation and identification from cervical and lumbar spinal X-rays.

Accurate annotation of vertebral bodies is crucial for automating the analysis of spinal X-ray images. However, manual annotation of these structures is a laborious and costly process due to their complex nature, including small sizes and varying shapes. To address this challenge and expedite the annotation process, we propose an ensemble pipeline called VertXNet. This pipeline currently combines two segmentation mechanisms, semantic segmentation using U-Net, and instance segmentation using Mask R-CNN, to automatically segment and label vertebral bodies in lateral cervical and lumbar spinal X-ray images. VertXNet enhances its effectiveness by adopting a rule-based strategy (termed the ensemble rule) for effectively combining segmentation outcomes from U-Net and Mask R-CNN. It determines vertebral body labels by recognizing specific reference vertebral instances, such as cervical vertebra 2 ('C2') in cervical spine X-rays and sacral vertebra 1 ('S1') in lumbar spine X-rays. Those references are commonly relatively easy to identify at the edge of the spine. To assess the performance of our proposed pipeline, we conducted evaluations on three spinal X-ray datasets, including two in-house datasets and one publicly available dataset. The ground truth annotations were provided by radiologists for comparison. Our experimental results have shown that the proposed pipeline outperformed two state-of-the-art (SOTA) segmentation models on our test dataset with a mean Dice of 0.90, vs. a mean Dice of 0.73 for Mask R-CNN and 0.72 for U-Net. We also demonstrated that VertXNet is a modular pipeline that enables using other SOTA model, like nnU-Net to further improve its performance. Furthermore, to evaluate the generalization ability of VertXNet on spinal X-rays, we directly tested the pre-trained pipeline on two additional datasets. A consistently strong performance was observed, with mean Dice coefficients of 0.89 and 0.88, respectively. In summary, VertXNet demonstrated significantly improved performance in vertebral body segmentation and labeling for spinal X-ray imaging. Its robustness and generalization were presented through the evaluation of both in-house clinical trial data and publicly available datasets.

Residual Aligner-based Network (RAN): Motion-separable structure for coarse-to-fine discontinuous deformable registration.

Deformable image registration, the estimation of the spatial transformation between different images, is an important task in medical imaging. Deep learning techniques have been shown to perform 3D image registration efficiently. However, current registration strategies often only focus on the deformation smoothness, which leads to the ignorance of complicated motion patterns (e.g., separate or sliding motions), especially for the intersection of organs. Thus, the performance when dealing with the discontinuous motions of multiple nearby objects is limited, causing undesired predictive outcomes in clinical usage, such as misidentification and mislocalization of lesions or other abnormalities. Consequently, we proposed a novel registration method to address this issue: a new Motion Separable backbone is exploited to capture the separate motion, with a theoretical analysis of the upper bound of the motions' discontinuity provided. In addition, a novel Residual Aligner module was used to disentangle and refine the predicted motions across the multiple neighboring objects/organs. We evaluate our method, Residual Aligner-based Network (RAN), on abdominal Computed Tomography (CT) scans and it has shown to achieve one of the most accurate unsupervised inter-subject registration for the 9 organs, with the highest-ranked registration of the veins (Dice Similarity Coefficient (%)/Average surface distance (mm): 62%/4.9mm for the vena cava and 34%/7.9mm for the portal and splenic vein), with a smaller model structure and less computation compared to state-of-the-art methods. Furthermore, when applied to lung CT, the RAN achieves comparable results to the best-ranked networks (94%/3.0mm), also with fewer parameters and less computation.

Normative spatiotemporal fetal brain maturation with satisfactory development at 2 years.

Maturation of the human fetal brain should follow precisely scheduled structural growth and folding of the cerebral cortex for optimal postnatal function1. We present a normative digital atlas of fetal brain maturation based on a prospective international cohort of healthy pregnant women2, selected using World Health Organization recommendations for growth standards3. Their fetuses were accurately dated in the first trimester, with satisfactory growth and neurodevelopment from early pregnancy to 2 years of age4,5. The atlas was produced using 1,059 optimal quality, three-dimensional ultrasound brain volumes from 899 of the fetuses and an automated analysis pipeline6-8. The atlas corresponds structurally to published magnetic resonance images9, but with finer anatomical details in deep grey matter. The between-study site variability represented less than 8.0% of the total variance of all brain measures, supporting pooling data from the eight study sites to produce patterns of normative maturation. We have thereby generated an average representation of each cerebral hemisphere between 14 and 31 weeks' gestation with quantification of intracranial volume variability and growth patterns. Emergent asymmetries were detectable from as early as 14 weeks, with peak asymmetries in regions associated with language development and functional lateralization between 20 and 26 weeks' gestation. These patterns were validated in 1,487 three-dimensional brain volumes from 1,295 different fetuses in the same cohort. We provide a unique spatiotemporal benchmark of fetal brain maturation from a large cohort with normative postnatal growth and neurodevelopment.

Accurate volume alignment of arbitrarily oriented tibiae based on a mutual attention network for osteoarthritis analysis.

Damage to cartilage is an important indicator of osteoarthritis progression, but manual extraction of cartilage morphology is time-consuming and prone to error. To address this, we hypothesize that automatic labeling of cartilage can be achieved through the comparison of contrasted and non-contrasted Computer Tomography (CT). However, this is non-trivial as the pre-clinical volumes are at arbitrary starting poses due to the lack of standardized acquisition protocols. Thus, we propose an annotation-free deep learning method, D-net, for accurate and automatic alignment of pre- and post-contrasted cartilage CT volumes. D-Net is based on a novel mutual attention network structure to capture large-range translation and full-range rotation without the need for a prior pose template. CT volumes of mice tibiae are used for validation, with synthetic transformation for training and tested with real pre- and post-contrasted CT volumes. Analysis of Variance (ANOVA) was used to compare the different network structures. Our proposed method, D-net, achieves a Dice coefficient of 0.87, and significantly outperforms other state-of-the-art deep learning models, in the real-world alignment of 50 pairs of pre- and post-contrasted CT volumes when cascaded as a multi-stage network.

Evolving polarisation of infiltrating and alveolar macrophages in the lung during metastatic progression of melanoma suggests CCR1 as a therapeutic target.

Metastatic tumour progression is facilitated by tumour associated macrophages (TAMs) that enforce pro-tumour mechanisms and suppress immunity. In pulmonary metastases, it is unclear whether TAMs comprise tissue resident or infiltrating, recruited macrophages; and the different expression patterns of these TAMs are not well established. Using the mouse melanoma B16F10 model of experimental pulmonary metastasis, we show that infiltrating macrophages (IM) change their gene expression from an early pro-inflammatory to a later tumour promoting profile as the lesions grow. In contrast, resident alveolar macrophages (AM) maintain expression of crucial pro-inflammatory/anti-tumour genes with time. During metastatic growth, the pool of macrophages, which initially contains mainly alveolar macrophages, increasingly consists of infiltrating macrophages potentially facilitating metastasis progression. Blocking chemokine receptor mediated macrophage infiltration in the lung revealed a prominent role for CCR2 in Ly6C+ pro-inflammatory monocyte/macrophage recruitment during metastasis progression, while inhibition of CCR2 signalling led to increased metastatic colony burden. CCR1 blockade, in contrast, suppressed late phase pro-tumour MR+Ly6C- monocyte/macrophage infiltration accompanied by expansion of the alveolar macrophage compartment and accumulation of NK cells, leading to reduced metastatic burden. These data indicate that IM has greater plasticity and higher phenotypic responsiveness to tumour challenge than AM. A considerable difference is also confirmed between CCR1 and CCR2 with regard to the recruited IM subsets, with CCR1 presenting a potential therapeutic target in pulmonary metastasis from melanoma.

BEAN: Brain Extraction and Alignment Network for 3D Fetal Neurosonography.

Brain extraction (masking of extra-cerebral tissues) and alignment are fundamental first steps of most neuroimage analysis pipelines. The lack of automated solutions for 3D ultrasound (US) has therefore limited its potential as a neuroimaging modality for studying fetal brain development using routinely acquired scans. In this work, we propose a convolutional neural network (CNN) that accurately and consistently aligns and extracts the fetal brain from minimally pre-processed 3D US scans. Our multi-task CNN, Brain Extraction and Alignment Network (BEAN), consists of two independent branches: 1) a fully-convolutional encoder-decoder branch for brain extraction of unaligned scans, and 2) a two-step regression-based branch for similarity alignment of the brain to a common coordinate space. BEAN was tested on 356 fetal head 3D scans spanning the gestational range of 14 to 30 weeks, significantly outperforming all current alternatives for fetal brain extraction and alignment. BEAN achieved state-of-the-art performance for both tasks, with a mean Dice Similarity Coefficient (DSC) of 0.94 for the brain extraction masks, and a mean DSC of 0.93 for the alignment of the target brain masks. The presented experimental results show that brain structures such as the thalamus, choroid plexus, cavum septum pellucidum, and Sylvian fissure, are consistently aligned throughout the dataset and remain clearly visible when the scans are averaged together. The BEAN implementation and related code can be found under www.github.com/felipemoser/kelluwen.

MOCOnet: Robust Motion Correction of Cardiovascular Magnetic Resonance T1 Mapping Using Convolutional Neural Networks.

Background: Quantitative cardiovascular magnetic resonance (CMR) T1 mapping has shown promise for advanced tissue characterisation in routine clinical practise. However, T1 mapping is prone to motion artefacts, which affects its robustness and clinical interpretation. Current methods for motion correction on T1 mapping are model-driven with no guarantee on generalisability, limiting its widespread use. In contrast, emerging data-driven deep learning approaches have shown good performance in general image registration tasks. We propose MOCOnet, a convolutional neural network solution, for generalisable motion artefact correction in T1 maps. Methods: The network architecture employs U-Net for producing distance vector fields and utilises warping layers to apply deformation to the feature maps in a coarse-to-fine manner. Using the UK Biobank imaging dataset scanned at 1.5T, MOCOnet was trained on 1,536 mid-ventricular T1 maps (acquired using the ShMOLLI method) with motion artefacts, generated by a customised deformation procedure, and tested on a different set of 200 samples with a diverse range of motion. MOCOnet was compared to a well-validated baseline multi-modal image registration method. Motion reduction was visually assessed by 3 human experts, with motion scores ranging from 0% (strictly no motion) to 100% (very severe motion). Results: MOCOnet achieved fast image registration (<1 second per T1 map) and successfully suppressed a wide range of motion artefacts. MOCOnet significantly reduced motion scores from 37.1±21.5 to 13.3±10.5 (p < 0.001), whereas the baseline method reduced it to 15.8±15.6 (p < 0.001). MOCOnet was significantly better than the baseline method in suppressing motion artefacts and more consistently (p = 0.007). Conclusion: MOCOnet demonstrated significantly better motion correction performance compared to a traditional image registration approach. Salvaging data affected by motion with robustness and in a time-efficient manner may enable better image quality and reliable images for immediate clinical interpretation.

Tumour subregion analysis of colorectal liver metastases using semi-automated clustering based on DCE-MRI: Comparison with histological subregions and impact on pharmacokinetic parameter analysis.

PURPOSE: To use a novel segmentation methodology based on dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) to define tumour subregions of liver metastases from colorectal cancer (CRC), to compare these with histology, and to use these to compare extracted pharmacokinetic (PK) parameters between tumour subregions. MATERIALS AND METHODS: This ethically-approved prospective study recruited patients with CRC and ≥1 hepatic metastases scheduled for hepatic resection. Patients underwent DCE-MRI pre-metastasectomy. Histological sections of resection specimens were spatially matched to DCE-MRI acquisitions and used to define histological subregions of viable and non-viable tumour. A semi-automated voxel-wise image segmentation algorithm based on the DCE-MRI contrast-uptake curves was used to define imaging subregions of viable and non-viable tumour. Overlap of histologically-defined and imaging subregions was compared using the Dice similarity coefficient (DSC). DCE-MRI PK parameters were compared for the whole tumour and histology-defined and imaging-derived subregions. RESULTS: Fourteen patients were included in the analysis. Direct histological comparison with imaging was possible in nine patients. Mean DSC for viable tumour subregions defined by imaging and histology was 0.738 (range 0.540-0.930). There were significant differences between Ktrans and kep for viable and non-viable subregions (p < 0.001) and between whole lesions and viable subregions (p < 0.001). CONCLUSION: We demonstrate good concordance of viable tumour segmentation based on pre-operative DCE-MRI with a post-operative histological gold-standard. This can be used to extract viable tumour-specific values from quantitative image analysis, and could improve treatment response assessment in clinical practice.

Image-Based Artefact Removal in Laser Scanning Microscopy.

Recent developments in laser scanning microscopy have greatly extended its applicability in cancer imaging beyond the visualization of complex biology, and opened up the possibility of quantitative analysis of inherently dynamic biological processes. However, the physics of image acquisition intrinsically means that image quality is subject to a tradeoff between a number of imaging parameters, including resolution, signal-to-noise ratio, and acquisition speed. We address the problem of geometric distortion, in particular, jaggedness artefacts that are caused by the variable motion of the microscope laser, by using a combination of image processing techniques. Image restoration methods have already shown great potential for post-acquisition image analysis. The performance of our proposed image restoration technique was first quantitatively evaluated using phantom data with different textures, and then qualitatively assessed using in vivo biological imaging data. In both cases, the presented method, comprising a combination of image registration and filtering, is demonstrated to have substantial improvement over state-of-the-art microscopy acquisition methods.

Patch-based lung ventilation estimation using multi-layer supervoxels.

Patch-based approaches have received substantial attention over the recent years in medical imaging. One of their potential applications may be to provide more anatomically consistent ventilation maps estimated on dynamic lung CT. An assessment of regional lung function may act as a guide for radiotherapy, ensuring a more accurate treatment plan. This in turn, could spare well-functioning parts of the lungs. We present a novel method for lung ventilation estimation from dynamic lung CT imaging, combining a supervoxel-based image representation with deformations estimated during deformable image registration, performed between peak breathing phases. For this we propose a method that tracks changes of the intensity of previously extracted supervoxels. For the evaluation of the method we calculate correlation of the estimated ventilation maps with static ventilation images acquired from hyperpolarized Xenon129 MRI. We also investigate the influence of different image registration methods used to estimate deformations between the peak breathing phases in the dynamic CT imaging. We show that our method performs favorably to other ventilation estimation methods commonly used in the field, independently of the image registration method applied to dynamic CT. Due to the patch-based approach of our method, it may be physiologically more consistent with lung anatomy than previous methods relying on voxel-wise relationships. In our method the ventilation is estimated for supervoxels, which tend to group spatially close voxels with similar intensity values. The proposed method was evaluated on a dataset consisting of three lung cancer patients undergoing radiotherapy treatment, and this resulted in a correlation of 0.485 with XeMRI ventilation images, compared with 0.393 for the intensity-based approach, 0.231 for the Jacobian-based method and 0.386 for the Hounsfield units averaging method, on average. Within the limitation of the small number of cases analyzed, results suggest that the presented technique may be advantageous for CT-based ventilation estimation. The results showing higher values of correlation of the proposed method demonstrate the potential of our method to more accurately mimic the lung physiology.

Evaluation of 2D and 3D ultrasound tracking algorithms and impact on ultrasound-guided liver radiotherapy margins.

PURPOSE: Compensation for respiratory motion is important during abdominal cancer treatments. In this work we report the results of the 2015 MICCAI Challenge on Liver Ultrasound Tracking and extend the 2D results to relate them to clinical relevance in form of reducing treatment margins and hence sparing healthy tissues, while maintaining full duty cycle. METHODS: We describe methodologies for estimating and temporally predicting respiratory liver motion from continuous ultrasound imaging, used during ultrasound-guided radiation therapy. Furthermore, we investigated the trade-off between tracking accuracy and runtime in combination with temporal prediction strategies and their impact on treatment margins. RESULTS: Based on 2D ultrasound sequences from 39 volunteers, a mean tracking accuracy of 0.9 mm was achieved when combining the results from the 4 challenge submissions (1.2 to 3.3 mm). The two submissions for the 3D sequences from 14 volunteers provided mean accuracies of 1.7 and 1.8 mm. In combination with temporal prediction, using the faster (41 vs 228 ms) but less accurate (1.4 vs 0.9 mm) tracking method resulted in substantially reduced treatment margins (70% vs 39%) in contrast to mid-ventilation margins, as it avoided non-linear temporal prediction by keeping the treatment system latency low (150 vs 400 ms). Acceleration of the best tracking method would improve the margin reduction to 75%. CONCLUSIONS: Liver motion estimation and prediction during free-breathing from 2D ultrasound images can substantially reduce the in-plane motion uncertainty and hence treatment margins. Employing an accurate tracking method while avoiding non-linear temporal prediction would be favorable. This approach has the potential to shorten treatment time compared to breath-hold and gated approaches, and increase treatment efficiency and safety.

Whole tumor kinetics analysis of 18F-fluoromisonidazole dynamic PET scans of non-small cell lung cancer patients, and correlations with perfusion CT blood flow.

BACKGROUND: To determine the relative abilities of compartment models to describe time-courses of 18F-fluoromisonidazole (FMISO) tumor uptake in patients with advanced stage non-small cell lung cancer (NSCLC) imaged using dynamic positron emission tomography (dPET), and study correlations between values of the blood flow-related parameter K1 obtained from fits of the models and an independent blood flow measure obtained from perfusion CT (pCT). NSCLC patients had a 45-min dynamic FMISO PET/CT scan followed by two static PET/CT acquisitions at 2 and 4-h post-injection. Perfusion CT scanning was then performed consisting of a 45-s cine CT. Reversible and irreversible two-, three- and four-tissue compartment models were fitted to 30 time-activity-curves (TACs) obtained for 15 whole tumor structures in 9 patients, each imaged twice. Descriptions of the TACs provided by the models were compared using the Akaike and Bayesian information criteria (AIC and BIC) and leave-one-out cross-validation. The precision with which fitted model parameters estimated ground-truth uptake kinetics was determined using statistical simulation techniques. Blood flow from pCT was correlated with K1 from PET kinetic models in addition to FMISO uptake levels. RESULTS: An irreversible three-tissue compartment model provided the best description of whole tumor FMISO uptake time-courses according to AIC, BIC, and cross-validation scores totaled across the TACs. The simulation study indicated that this model also provided more precise estimates of FMISO uptake kinetics than other two- and three-tissue models. The K1 values obtained from fits of the irreversible three-tissue model correlated strongly with independent blood flow measurements obtained from pCT (Pearson r coefficient = 0.81). The correlation from the irreversible three-tissue model (r = 0.81) was stronger than that from than K1 values obtained from fits of a two-tissue compartment model (r = 0.68), or FMISO uptake levels in static images taken at time-points from tracer injection through to 4 h later (maximum at 2 min, r = 0.70). CONCLUSIONS: Time-courses of whole tumor FMISO uptake by advanced stage NSCLC are described best by an irreversible three-tissue compartment model. The K1 values obtained from fits of the irreversible three-tissue model correlated strongly with independent blood flow measurements obtained from perfusion CT (r = 0.81).

Functional Parameters Derived from Magnetic Resonance Imaging Reflect Vascular Morphology in Preclinical Tumors and in Human Liver Metastases.

Purpose: Tumor vessels influence the growth and response of tumors to therapy. Imaging vascular changes in vivo using dynamic contrast-enhanced MRI (DCE-MRI) has shown potential to guide clinical decision making for treatment. However, quantitative MR imaging biomarkers of vascular function have not been widely adopted, partly because their relationship to structural changes in vessels remains unclear. We aimed to elucidate the relationships between vessel function and morphology in vivo Experimental Design: Untreated preclinical tumors with different levels of vascularization were imaged sequentially using DCE-MRI and CT. Relationships between functional parameters from MR (iAUC, K trans, and BATfrac) and structural parameters from CT (vessel volume, radius, and tortuosity) were assessed using linear models. Tumors treated with anti-VEGFR2 antibody were then imaged to determine whether antiangiogenic therapy altered these relationships. Finally, functional-structural relationships were measured in 10 patients with liver metastases from colorectal cancer.Results: Functional parameters iAUC and K trans primarily reflected vessel volume in untreated preclinical tumors. The relationships varied spatially and with tumor vascularity, and were altered by antiangiogenic treatment. In human liver metastases, all three structural parameters were linearly correlated with iAUC and K trans For iAUC, structural parameters also modified each other's effect.Conclusions: Our findings suggest that MR imaging biomarkers of vascular function are linked to structural changes in tumor vessels and that antiangiogenic therapy can affect this link. Our work also demonstrates the feasibility of three-dimensional functional-structural validation of MR biomarkers in vivo to improve their biological interpretation and clinical utility. Clin Cancer Res; 24(19); 4694-704. ©2018 AACR.

GIFTed Demons: deformable image registration with local structure-preserving regularization using supervoxels for liver applications.

Deformable image registration, a key component of motion correction in medical imaging, needs to be efficient and provides plausible spatial transformations that reliably approximate biological aspects of complex human organ motion. Standard approaches, such as Demons registration, mostly use Gaussian regularization for organ motion, which, though computationally efficient, rule out their application to intrinsically more complex organ motions, such as sliding interfaces. We propose regularization of motion based on supervoxels, which provides an integrated discontinuity preserving prior for motions, such as sliding. More precisely, we replace Gaussian smoothing by fast, structure-preserving, guided filtering to provide efficient, locally adaptive regularization of the estimated displacement field. We illustrate the approach by applying it to estimate sliding motions at lung and liver interfaces on challenging four-dimensional computed tomography (CT) and dynamic contrast-enhanced magnetic resonance imaging datasets. The results show that guided filter-based regularization improves the accuracy of lung and liver motion correction as compared to Gaussian smoothing. Furthermore, our framework achieves state-of-the-art results on a publicly available CT liver dataset.

A level-set approach to joint image segmentation and registration with application to CT lung imaging.

Automated analysis of structural imaging such as lung Computed Tomography (CT) plays an increasingly important role in medical imaging applications. Despite significant progress in the development of image registration and segmentation methods, lung registration and segmentation remain a challenging task. In this paper, we present a novel image registration and segmentation approach, for which we develop a new mathematical formulation to jointly segment and register three-dimensional lung CT volumes. The new algorithm is based on a level-set formulation, which merges a classic Chan-Vese segmentation with the active dense displacement field estimation. Combining registration with segmentation has two key advantages: it allows to eliminate the problem of initializing surface based segmentation methods, and to incorporate prior knowledge into the registration in a mathematically justified manner, while remaining computationally attractive. We evaluate our framework on a publicly available lung CT data set to demonstrate the properties of the new formulation. The presented results show the improved accuracy for our joint segmentation and registration algorithm when compared to registration and segmentation performed separately.

Supervoxels for Graph Cuts-Based Deformable Image Registration Using Guided Image Filtering.

In this work we propose to combine a supervoxel-based image representation with the concept of graph cuts as an efficient optimization technique for 3D deformable image registration. Due to the pixels/voxels-wise graph construction, the use of graph cuts in this context has been mainly limited to 2D applications. However, our work overcomes some of the previous limitations by posing the problem on a graph created by adjacent supervoxels, where the number of nodes in the graph is reduced from the number of voxels to the number of supervoxels. We demonstrate how a supervoxel image representation, combined with graph cuts-based optimization can be applied to 3D data. We further show that the application of a relaxed graph representation of the image, followed by guided image filtering over the estimated deformation field, allows us to model 'sliding motion'. Applying this method to lung image registration, results in highly accurate image registration and anatomically plausible estimations of the deformations. Evaluation of our method on a publicly available Computed Tomography lung image dataset (www.dir-lab.com) leads to the observation that our new approach compares very favorably with state-of-the-art in continuous and discrete image registration methods achieving Target Registration Error of 1.16mm on average per landmark.

Advances and challenges in deformable image registration: From image fusion to complex motion modelling.

Over the past 20 years, the field of medical image registration has significantly advanced from multi-modal image fusion to highly non-linear, deformable image registration for a wide range of medical applications and imaging modalities, involving the compensation and analysis of physiological organ motion or of tissue changes due to growth or disease patterns. While the original focus of image registration has predominantly been on correcting for rigid-body motion of brain image volumes acquired at different scanning sessions, often with different modalities, the advent of dedicated longitudinal and cross-sectional brain studies soon necessitated the development of more sophisticated methods that are able to detect and measure local structural or functional changes, or group differences. Moving outside of the brain, cine imaging and dynamic imaging required the development of deformable image registration to directly measure or compensate for local tissue motion. Since then, deformable image registration has become a general enabling technology. In this work we will present our own contributions to the state-of-the-art in deformable multi-modal fusion and complex motion modelling, and then discuss remaining challenges and provide future perspectives to the field.

Pieces-of-parts for supervoxel segmentation with global context: Application to DCE-MRI tumour delineation.

Rectal tumour segmentation in dynamic contrast-enhanced MRI (DCE-MRI) is a challenging task, and an automated and consistent method would be highly desirable to improve the modelling and prediction of patient outcomes from tissue contrast enhancement characteristics - particularly in routine clinical practice. A framework is developed to automate DCE-MRI tumour segmentation, by introducing: perfusion-supervoxels to over-segment and classify DCE-MRI volumes using the dynamic contrast enhancement characteristics; and the pieces-of-parts graphical model, which adds global (anatomic) constraints that further refine the supervoxel components that comprise the tumour. The framework was evaluated on 23 DCE-MRI scans of patients with rectal adenocarcinomas, and achieved a voxelwise area-under the receiver operating characteristic curve (AUC) of 0.97 compared to expert delineations. Creating a binary tumour segmentation, 21 of the 23 cases were segmented correctly with a median Dice similarity coefficient (DSC) of 0.63, which is close to the inter-rater variability of this challenging task. A second study is also included to demonstrate the method's generalisability and achieved a DSC of 0.71. The framework achieves promising results for the underexplored area of rectal tumour segmentation in DCE-MRI, and the methods have potential to be applied to other DCE-MRI and supervoxel segmentation problems.

Deformable image registration by combining uncertainty estimates from supervoxel belief propagation.

Discrete optimisation strategies have a number of advantages over their continuous counterparts for deformable registration of medical images. For example: it is not necessary to compute derivatives of the similarity term; dense sampling of the search space reduces the risk of becoming trapped in local optima; and (in principle) an optimum can be found without resorting to iterative coarse-to-fine warping strategies. However, the large complexity of high-dimensional medical data renders a direct voxel-wise estimation of deformation vectors impractical. For this reason, previous work on medical image registration using graphical models has largely relied on using a parameterised deformation model and on the use of iterative coarse-to-fine optimisation schemes. In this paper, we propose an approach that enables accurate voxel-wise deformable registration of high-resolution 3D images without the need for intermediate image warping or a multi-resolution scheme. This is achieved by representing the image domain as multiple comprehensive supervoxel layers and making use of the full marginal distribution of all probable displacement vectors after inferring regularity of the deformations using belief propagation. The optimisation acts on the coarse scale representation of supervoxels, which provides sufficient spatial context and is robust to noise in low contrast areas. Minimum spanning trees, which connect neighbouring supervoxels, are employed to model pair-wise deformation dependencies. The optimal displacement for each voxel is calculated by considering the probabilities for all displacements over all overlapping supervoxel graphs and subsequently seeking the mode of this distribution. We demonstrate the applicability of this concept for two challenging applications: first, for intra-patient motion estimation in lung CT scans; and second, for atlas-based segmentation propagation of MRI brain scans. For lung registration, the voxel-wise mode of displacements is found using the mean-shift algorithm, which enables us to determine continuous valued sub-voxel motion vectors. Finding the mode of brain segmentation labels is performed using a voxel-wise majority voting weighted by the displacement uncertainty estimates. Our experimental results show significant improvements in registration accuracy when using the additional information provided by the registration uncertainty estimates. The multi-layer approach enables fusion of multiple complementary proposals, extending the popular fusion approaches from multi-image registration to probabilistic one-to-one image registration.

Multispectral image registration based on local canonical correlation analysis.

Medical scans are today routinely acquired using multiple sequences or contrast settings, resulting in multispectral data. For the automatic analysis of this data, the evaluation of multispectral similarity is essential. So far, few concepts have been proposed to deal in a principled way with images containing multiple channels. Here, we present a new approach based on a well known statistical technique: canonical correlation analysis (CCA). CCA finds a mapping of two multidimensional variables into two new bases, which best represent the true underlying relations of the signals. In contrast to previously used metrics, it is therefore able to find new correlations based on linear combinations of multiple channels. We extend this concept to efficiently model local canonical correlation (LCCA) between image patches. This novel, more general similarity metric can be applied to images with an arbitrary number of channels. The most important property of LCCA is its invariance to affine transformations of variables. When used on local histograms, LCCA can also deal with multimodal similarity. We demonstrate the performance of our concept on challenging clinical multispectral datasets.