Deep learning links localized digital pathology phenotypes with transcriptional subtype and patient outcome in glioblastoma.

BACKGROUND: Deep learning has revolutionized medical image analysis in cancer pathology, where it had a substantial clinical impact by supporting the diagnosis and prognostic rating of cancer. Among the first available digital resources in the field of brain cancer is glioblastoma, the most common and fatal brain cancer. At the histologic level, glioblastoma is characterized by abundant phenotypic variability that is poorly linked with patient prognosis. At the transcriptional level, 3 molecular subtypes are distinguished with mesenchymal-subtype tumors being associated with increased immune cell infiltration and worse outcome. RESULTS: We address genotype-phenotype correlations by applying an Xception convolutional neural network to a discovery set of 276 digital hematozylin and eosin (H&E) slides with molecular subtype annotation and an independent The Cancer Genome Atlas-based validation cohort of 178 cases. Using this approach, we achieve high accuracy in H&E-based mapping of molecular subtypes (area under the curve for classical, mesenchymal, and proneural = 0.84, 0.81, and 0.71, respectively; P < 0.001) and regions associated with worse outcome (univariable survival model P < 0.001, multivariable P = 0.01). The latter were characterized by higher tumor cell density (P < 0.001), phenotypic variability of tumor cells (P < 0.001), and decreased T-cell infiltration (P = 0.017). CONCLUSIONS: We modify a well-known convolutional neural network architecture for glioblastoma digital slides to accurately map the spatial distribution of transcriptional subtypes and regions predictive of worse outcome, thereby showcasing the relevance of artificial intelligence-enabled image mining in brain cancer.

Fully automated segmentation of left ventricular scar from 3D late gadolinium enhancement magnetic resonance imaging using a cascaded multi-planar U-Net (CMPU-Net).

PURPOSE: Three-dimensional (3D) late gadolinium enhancement magnetic resonance (LGE-MR) imaging enables the quantification of myocardial scar at high resolution with unprecedented volumetric visualization. Automated segmentation of myocardial scar is critical for the potential clinical translation of this technique given the number of tomographic images acquired. METHODS: In this paper, we describe the development of cascaded multi-planar U-Net (CMPU-Net) to efficiently segment the boundary of the left ventricle (LV) myocardium and scar from 3D LGE-MR images. In this approach, two subnets, each containing three U-Nets, were cascaded to first segment the LV myocardium and then segment the scar within the presegmented LV myocardium. The U-Nets were trained separately using two-dimensional (2D) slices extracted from axial, sagittal, and coronal slices of 3D LGE-MR images. We used 3D LGE-MR images from 34 subjects with chronic ischemic cardiomyopathy. The U-Nets were trained using 8430 slices, extracted in three orthogonal directions from 18 images. In the testing phase, the outputs of U-Nets of each subnet were combined using the majority voting system for final label prediction of each voxel in the image. The developed method was tested for accuracy by comparing its results to manual segmentations of LV myocardium and LV scar from 7250 slices extracted from 16 3D LGE-MR images. Our method was also compared to numerous alternative methods based on machine learning, energy minimization, and intensity-thresholds. RESULTS: Our algorithm reported a mean dice similarity coefficient (DSC), absolute volume difference (AVD), and Hausdorff distance (HD) of 85.14% ± 3.36%, 43.72 ± 27.18 cm3 , and 19.21 ± 4.74 mm for determining the boundaries of LV myocardium from LGE-MR images. Our method also yielded a mean DSC, AVD, and HD of 88.61% ± 2.54%, 9.33 ± 7.24 cm3 , and 17.04 ± 9.93 mm for LV scar segmentation on the unobserved test dataset. Our method significantly outperformed the alternative techniques in segmentation accuracy (P < 0.05). CONCLUSIONS: The CMPU-Net method provided fully automated segmentation of LV scar from 3D LGE-MR images and outperformed the alternative techniques.

Weakly Supervised Estimation of Shadow Confidence Maps in Fetal Ultrasound Imaging.

Detecting acoustic shadows in ultrasound images is important in many clinical and engineering applications. Real-time feedback of acoustic shadows can guide sonographers to a standardized diagnostic viewing plane with minimal artifacts and can provide additional information for other automatic image analysis algorithms. However, automatically detecting shadow regions using learning-based algorithms is challenging because pixel-wise ground truth annotation of acoustic shadows is subjective and time consuming. In this paper, we propose a weakly supervised method for automatic confidence estimation of acoustic shadow regions. Our method is able to generate a dense shadow-focused confidence map. In our method, a shadow-seg module is built to learn general shadow features for shadow segmentation, based on global image-level annotations as well as a small number of coarse pixel-wise shadow annotations. A transfer function is introduced to extend the obtained binary shadow segmentation to a reference confidence map. In addition, a confidence estimation network is proposed to learn the mapping between input images and the reference confidence maps. This network is able to predict shadow confidence maps directly from input images during inference. We use evaluation metrics such as DICE, inter-class correlation, and so on, to verify the effectiveness of our method. Our method is more consistent than human annotation and outperforms the state-of-the-art quantitatively in shadow segmentation and qualitatively in confidence estimation of shadow regions. Furthermore, we demonstrate the applicability of our method by integrating shadow confidence maps into tasks such as ultrasound image classification, multi-view image fusion, and automated biometric measurements.

Automated cardiovascular magnetic resonance image analysis with fully convolutional networks.

BACKGROUND: Cardiovascular resonance (CMR) imaging is a standard imaging modality for assessing cardiovascular diseases (CVDs), the leading cause of death globally. CMR enables accurate quantification of the cardiac chamber volume, ejection fraction and myocardial mass, providing information for diagnosis and monitoring of CVDs. However, for years, clinicians have been relying on manual approaches for CMR image analysis, which is time consuming and prone to subjective errors. It is a major clinical challenge to automatically derive quantitative and clinically relevant information from CMR images. METHODS: Deep neural networks have shown a great potential in image pattern recognition and segmentation for a variety of tasks. Here we demonstrate an automated analysis method for CMR images, which is based on a fully convolutional network (FCN). The network is trained and evaluated on a large-scale dataset from the UK Biobank, consisting of 4,875 subjects with 93,500 pixelwise annotated images. The performance of the method has been evaluated using a number of technical metrics, including the Dice metric, mean contour distance and Hausdorff distance, as well as clinically relevant measures, including left ventricle (LV) end-diastolic volume (LVEDV) and end-systolic volume (LVESV), LV mass (LVM); right ventricle (RV) end-diastolic volume (RVEDV) and end-systolic volume (RVESV). RESULTS: By combining FCN with a large-scale annotated dataset, the proposed automated method achieves a high performance in segmenting the LV and RV on short-axis CMR images and the left atrium (LA) and right atrium (RA) on long-axis CMR images. On a short-axis image test set of 600 subjects, it achieves an average Dice metric of 0.94 for the LV cavity, 0.88 for the LV myocardium and 0.90 for the RV cavity. The mean absolute difference between automated measurement and manual measurement is 6.1 mL for LVEDV, 5.3 mL for LVESV, 6.9 gram for LVM, 8.5 mL for RVEDV and 7.2 mL for RVESV. On long-axis image test sets, the average Dice metric is 0.93 for the LA cavity (2-chamber view), 0.95 for the LA cavity (4-chamber view) and 0.96 for the RA cavity (4-chamber view). The performance is comparable to human inter-observer variability. CONCLUSIONS: We show that an automated method achieves a performance on par with human experts in analysing CMR images and deriving clinically relevant measures.

Multi-Atlas Segmentation Using Partially Annotated Data: Methods and Annotation Strategies.

Multi-atlas segmentation is a widely used tool in medical image analysis, providing robust and accurate results by learning from annotated atlas datasets. However, the availability of fully annotated atlas images for training is limited due to the time required for the labelling task. Segmentation methods requiring only a proportion of each atlas image to be labelled could therefore reduce the workload on expert raters tasked with annotating atlas images. To address this issue, we first re-examine the labelling problem common in many existing approaches and formulate its solution in terms of a Markov Random Field energy minimisation problem on a graph connecting atlases and the target image. This provides a unifying framework for multi-atlas segmentation. We then show how modifications in the graph configuration of the proposed framework enable the use of partially annotated atlas images and investigate different partial annotation strategies. The proposed method was evaluated on two Magnetic Resonance Imaging (MRI) datasets for hippocampal and cardiac segmentation. Experiments were performed aimed at (1) recreating existing segmentation techniques with the proposed framework and (2) demonstrating the potential of employing sparsely annotated atlas data for multi-atlas segmentation.

Metric learning with spectral graph convolutions on brain connectivity networks.

Graph representations are often used to model structured data at an individual or population level and have numerous applications in pattern recognition problems. In the field of neuroscience, where such representations are commonly used to model structural or functional connectivity between a set of brain regions, graphs have proven to be of great importance. This is mainly due to the capability of revealing patterns related to brain development and disease, which were previously unknown. Evaluating similarity between these brain connectivity networks in a manner that accounts for the graph structure and is tailored for a particular application is, however, non-trivial. Most existing methods fail to accommodate the graph structure, discarding information that could be beneficial for further classification or regression analyses based on these similarities. We propose to learn a graph similarity metric using a siamese graph convolutional neural network (s-GCN) in a supervised setting. The proposed framework takes into consideration the graph structure for the evaluation of similarity between a pair of graphs, by employing spectral graph convolutions that allow the generalisation of traditional convolutions to irregular graphs and operates in the graph spectral domain. We apply the proposed model on two datasets: the challenging ABIDE database, which comprises functional MRI data of 403 patients with autism spectrum disorder (ASD) and 468 healthy controls aggregated from multiple acquisition sites, and a set of 2500 subjects from UK Biobank. We demonstrate the performance of the method for the tasks of classification between matching and non-matching graphs, as well as individual subject classification and manifold learning, showing that it leads to significantly improved results compared to traditional methods.

PVR: Patch-to-Volume Reconstruction for Large Area Motion Correction of Fetal MRI.

In this paper, we present a novel method for the correction of motion artifacts that are present in fetal magnetic resonance imaging (MRI) scans of the whole uterus. Contrary to current slice-to-volume registration (SVR) methods, requiring an inflexible anatomical enclosure of a single investigated organ, the proposed patch-to-volume reconstruction (PVR) approach is able to reconstruct a large field of view of non-rigidly deforming structures. It relaxes rigid motion assumptions by introducing a specific amount of redundant information that is exploited with parallelized patchwise optimization, super-resolution, and automatic outlier rejection. We further describe and provide an efficient parallel implementation of PVR allowing its execution within reasonable time on commercially available graphics processing units, enabling its use in the clinical practice. We evaluate PVR's computational overhead compared with standard methods and observe improved reconstruction accuracy in the presence of affine motion artifacts compared with conventional SVR in synthetic experiments. Furthermore, we have evaluated our method qualitatively and quantitatively on real fetal MRI data subject to maternal breathing and sudden fetal movements. We evaluate peak-signal-to-noise ratio, structural similarity index, and cross correlation with respect to the originally acquired data and provide a method for visual inspection of reconstruction uncertainty. We further evaluate the distance error for selected anatomical landmarks in the fetal head, as well as calculating the mean and maximum displacements resulting from automatic non-rigid registration to a motion-free ground truth image. These experiments demonstrate a successful application of PVR motion compensation to the whole fetal body, uterus, and placenta.

Contribution of mitral valve leaflet length and septal wall thickness to outflow tract obstruction in patients with hypertrophic cardiomyopathy.

We sought to examine whether elongation of the mitral valve leaflets in patients with hypertrophic cardiomyopathy (HCM) is synergistic to septal wall thickness (SWT) in the development of left ventricular outflow tract obstruction (LVOTO). HCM is a common genetic cardiac disease characterized by asymmetric septal hypertrophy and predisposition towards LVOTO. It has been reported that elongation of the mitral valve leaflets may be a primary phenotypic feature and contribute to LVOTO. However, the relative contribution of this finding versus SWT has not been studied. 152 patients (76 with HCM and 76 non-diseased age, race and BSA-matched controls) and 18 young, healthy volunteers were studied. SWT and the anterior mitral valve leaflet length (AMVLL) were measured using cine MRI. The combined contribution of these variables (SWT × AMVLL) was described as the Septal Anterior Leaflet Product (SALP). Peak LVOT pressure gradient was determined by Doppler interrogation and defined as "obstructive" if ≥ 30 mmHg. Patients with HCM were confirmed to have increased AMVLL compared with controls and volunteers (p < 0.01). Among HCM patients, both SWT and SALP were significantly higher in patients with LVOTO (N = 17) versus without. SALP showed modest improvement in predictive accuracy for LVOTO (AUC = 0.81) among the HCM population versus SWT alone (AUC = 0.77). However, in isolated patients this variable identified patients with LVOTO despite modest SWT. Elongation of the AMVLL is a primary phenotypic feature of HCM. While incremental contributions to LVOTO appear modest at a population level, specific patients may have dominant contribution to LVOTO. The combined marker of SALP allows for maintained identification of such patients despite modest increases in SWT.

Stratified Decision Forests for Accurate Anatomical Landmark Localization in Cardiac Images.

Accurate localization of anatomical landmarks is an important step in medical imaging, as it provides useful prior information for subsequent image analysis and acquisition methods. It is particularly useful for initialization of automatic image analysis tools (e.g. segmentation and registration) and detection of scan planes for automated image acquisition. Landmark localization has been commonly performed using learning based approaches, such as classifier and/or regressor models. However, trained models may not generalize well in heterogeneous datasets when the images contain large differences due to size, pose and shape variations of organs. To learn more data-adaptive and patient specific models, we propose a novel stratification based training model, and demonstrate its use in a decision forest. The proposed approach does not require any additional training information compared to the standard model training procedure and can be easily integrated into any decision tree framework. The proposed method is evaluated on 1080 3D high-resolution and 90 multi-stack 2D cardiac cine MR images. The experiments show that the proposed method achieves state-of-the-art landmark localization accuracy and outperforms standard regression and classification based approaches. Additionally, the proposed method is used in a multi-atlas segmentation to create a fully automatic segmentation pipeline, and the results show that it achieves state-of-the-art segmentation accuracy.

DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks.

In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known GrabCut [1] method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naïve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.

Optimization-based interactive segmentation interface for multiregion problems.

Interactive segmentation is becoming of increasing interest to the medical imaging community in that it combines the positive aspects of both manual and automated segmentation. However, general-purpose tools have been lacking in terms of segmenting multiple regions simultaneously with a high degree of coupling between groups of labels. Hierarchical max-flow segmentation has taken advantage of this coupling for individual applications, but until recently, these algorithms were constrained to a particular hierarchy and could not be considered general-purpose. In a generalized form, the hierarchy for any given segmentation problem is specified in run-time, allowing different hierarchies to be quickly explored. We present an interactive segmentation interface, which uses generalized hierarchical max-flow for optimization-based multiregion segmentation guided by user-defined seeds. Applications in cardiac and neonatal brain segmentation are given as example applications of its generality.

Hierarchical max-flow segmentation framework for multi-atlas segmentation with Kohonen self-organizing map based Gaussian mixture modeling.

The incorporation of intensity, spatial, and topological information into large-scale multi-region segmentation has been a topic of ongoing research in medical image analysis. Multi-region segmentation problems, such as segmentation of brain structures, pose unique challenges in image segmentation in which regions may not have a defined intensity, spatial, or topological distinction, but rely on a combination of the three. We propose a novel framework within the Advanced segmentation tools (ASETS)(2), which combines large-scale Gaussian mixture models trained via Kohonen self-organizing maps, with deformable registration, and a convex max-flow optimization algorithm incorporating region topology as a hierarchy or tree. Our framework is validated on two publicly available neuroimaging datasets, the OASIS and MRBrainS13 databases, against the more conventional Potts model, achieving more accurate segmentations. Each component is accelerated using general-purpose programming on graphics processing Units to ensure computational feasibility.

Image-based Reconstruction of 3D Myocardial Infarct Geometry for Patient Specific Applications.

Accurate reconstruction of the three-dimensional (3D) geometry of a myocardial infarct from two-dimensional (2D) multi-slice image sequences has important applications in the clinical evaluation and treatment of patients with ischemic cardiomyopathy. However, this reconstruction is challenging because the resolution of common clinical scans used to acquire infarct structure, such as short-axis, late-gadolinium enhanced cardiac magnetic resonance (LGE-CMR) images, is low, especially in the out-of-plane direction. In this study, we propose a novel technique to reconstruct the 3D infarct geometry from low resolution clinical images. Our methodology is based on a function called logarithm of odds (LogOdds), which allows the broader class of linear combinations in the LogOdds vector space as opposed to being limited to only a convex combination in the binary label space. To assess the efficacy of the method, we used high-resolution LGE-CMR images of 36 human hearts in vivo, and 3 canine hearts ex vivo. The infarct was manually segmented in each slice of the acquired images, and the manually segmented data were downsampled to clinical resolution. The developed method was then applied to the downsampled image slices, and the resulting reconstructions were compared with the manually segmented data. Several existing reconstruction techniques were also implemented, and compared with the proposed method. The results show that the LogOdds method significantly outperforms all the other tested methods in terms of region overlap.

Joint segmentation of lumen and outer wall from femoral artery MR images: Towards 3D imaging measurements of peripheral arterial disease.

Three-dimensional (3D) measurements of peripheral arterial disease (PAD) plaque burden extracted from fast black-blood magnetic resonance (MR) images have shown to be more predictive of clinical outcomes than PAD stenosis measurements. To this end, accurate segmentation of the femoral artery lumen and outer wall is required for generating volumetric measurements of PAD plaque burden. Here, we propose a semi-automated algorithm to jointly segment the femoral artery lumen and outer wall surfaces from 3D black-blood MR images, which are reoriented and reconstructed along the medial axis of the femoral artery to obtain improved spatial coherence between slices of the long, thin femoral artery and to reduce computation time. The developed segmentation algorithm enforces two priors in a global optimization manner: the spatial consistency between the adjacent 2D slices and the anatomical region order between the femoral artery lumen and outer wall surfaces. The formulated combinatorial optimization problem for segmentation is solved globally and exactly by means of convex relaxation using a coupled continuous max-flow (CCMF) model, which is a dual formulation to the convex relaxed optimization problem. In addition, the CCMF model directly derives an efficient duality-based algorithm based on the modern multiplier augmented optimization scheme, which has been implemented on a GPU for fast computation. The computed segmentations from the developed algorithm were compared to manual delineations from experts using 20 black-blood MR images. The developed algorithm yielded both high accuracy (Dice similarity coefficients ≥ 87% for both the lumen and outer wall surfaces) and high reproducibility (intra-class correlation coefficient of 0.95 for generating vessel wall area), while outperforming the state-of-the-art method in terms of computational time by a factor of ≈ 20.

Image-based reconstruction of three-dimensional myocardial infarct geometry for patient-specific modeling of cardiac electrophysiology.

PURPOSE: Accurate three-dimensional (3D) reconstruction of myocardial infarct geometry is crucial to patient-specific modeling of the heart aimed at providing therapeutic guidance in ischemic cardiomyopathy. However, myocardial infarct imaging is clinically performed using two-dimensional (2D) late-gadolinium enhanced cardiac magnetic resonance (LGE-CMR) techniques, and a method to build accurate 3D infarct reconstructions from the 2D LGE-CMR images has been lacking. The purpose of this study was to address this need. METHODS: The authors developed a novel methodology to reconstruct 3D infarct geometry from segmented low-resolution (Lo-res) clinical LGE-CMR images. Their methodology employed the so-called logarithm of odds (LogOdds) function to implicitly represent the shape of the infarct in segmented image slices as LogOdds maps. These 2D maps were then interpolated into a 3D image, and the result transformed via the inverse of LogOdds to a binary image representing the 3D infarct geometry. To assess the efficacy of this method, the authors utilized 39 high-resolution (Hi-res) LGE-CMR images, including 36 in vivo acquisitions of human subjects with prior myocardial infarction and 3 ex vivo scans of canine hearts following coronary ligation to induce infarction. The infarct was manually segmented by trained experts in each slice of the Hi-res images, and the segmented data were downsampled to typical clinical resolution. The proposed method was then used to reconstruct 3D infarct geometry from the downsampled images, and the resulting reconstructions were compared with the manually segmented data. The method was extensively evaluated using metrics based on geometry as well as results of electrophysiological simulations of cardiac sinus rhythm and ventricular tachycardia in individual hearts. Several alternative reconstruction techniques were also implemented and compared with the proposed method. RESULTS: The accuracy of the LogOdds method in reconstructing 3D infarct geometry, as measured by the Dice similarity coefficient, was 82.10% ± 6.58%, a significantly higher value than those of the alternative reconstruction methods. Among outcomes of electrophysiological simulations with infarct reconstructions generated by various methods, the simulation results corresponding to the LogOdds method showed the smallest deviation from those corresponding to the manual reconstructions, as measured by metrics based on both activation maps and pseudo-ECGs. CONCLUSIONS: The authors have developed a novel method for reconstructing 3D infarct geometry from segmented slices of Lo-res clinical 2D LGE-CMR images. This method outperformed alternative approaches in reproducing expert manual 3D reconstructions and in electrophysiological simulations.

Multi-atlas Segmentation as a Graph Labelling Problem: Application to Partially Annotated Atlas Data.

Manually annotating images for multi-atlas segmentation is an expensive and often limiting factor in reliable automated segmentation of large databases. Segmentation methods requiring only a proportion of each atlas image to be labelled could potentially reduce the workload on expert raters tasked with labelling images. However, exploiting such a database of partially labelled atlases is not possible with state-of-the-art multi-atlas segmentation methods. In this paper we revisit the problem of multi-atlas segmentation and formulate its solution in terms of graph-labelling. Our graphical approach uses a Markov Random Field (MRF) formulation of the problem and constructs a graph connecting atlases and the target image. This provides a unifying framework for label propagation. More importantly, the proposed method can be used for segmentation using only partially labelled atlases. We furthermore provide an extension to an existing continuous MRF optimisation method to solve the proposed problem formulation. We show that the proposed method, applied to hippocampal segmentation of 202 subjects from the ADNI database, remains robust and accurate even when the proportion of manually labelled slices in the atlases is reduced to 20%.

3D MR ventricle segmentation in pre-term infants with post-hemorrhagic ventricle dilatation (PHVD) using multi-phase geodesic level-sets.

Intraventricular hemorrhage (IVH) or bleed within the cerebral ventricles is a common condition among very low birth weight pre-term neonates. The prognosis for these patients is worsened should they develop progressive ventricular dilatation, i.e., post-hemorrhagic ventricle dilatation (PHVD), which occurs in 10-30% of IVH patients. Accurate measurement of ventricular volume would be valuable information and could be used to predict PHVD and determine whether that specific patient with ventricular dilatation requires treatment. While the monitoring of PHVD in infants is typically done by repeated transfontanell 2D ultrasound (US) and not MRI, once the patient's fontanels have closed around 12-18months of life, the follow-up patient scans are done by MRI. Manual segmentation of ventricles from MR images is still seen as a gold standard. However, it is extremely time- and labor-consuming, and it also has observer variability. This paper proposes an accurate multiphase geodesic level-set segmentation algorithm for the extraction of the cerebral ventricle system of pre-term PHVD neonates from 3D T1 weighted MR images. The proposed segmentation algorithm makes use of multi-region segmentation technique associated with spatial priors built from a multi-atlas registration scheme. The leave-one-out cross validation with 19 patients with mild enlargement of ventricles and 7 hydrocephalus patients shows that the proposed method is accurate, suggesting that the proposed approach could be potentially used for volumetric and morphological analysis of the ventricle system of IVH neonatal brains in clinical practice.

Globally optimal co-segmentation of three-dimensional pulmonary ¹H and hyperpolarized ³He MRI with spatial consistence prior.

Pulmonary imaging using hyperpolarized (3)He/(129)Xe gas is emerging as a new way to understand the regional nature of pulmonary ventilation abnormalities in obstructive lung diseases. However, the quantitative information derived is completely dependent on robust methods to segment both functional and structural/anatomical data. Here, we propose an approach to jointly segment the lung cavity from (1)H and (3)He pulmonary magnetic resonance images (MRI) by constraining the spatial consistency of the two segmentation regions, which simultaneously employs the image features from both modalities. We formulated the proposed co-segmentation problem as a coupled continuous min-cut model and showed that this combinatorial optimization problem can be solved globally and exactly by means of convex relaxation. In particular, we introduced a dual coupled continuous max-flow model to study the convex relaxed coupled continuous min-cut model under a primal and dual perspective. This gave rise to an efficient duality-based convex optimization algorithm. We implemented the proposed algorithm in parallel using general-purpose programming on graphics processing unit (GPGPU), which substantially increased its computational efficiency. Our experiments explored a clinical dataset of 25 subjects with chronic obstructive pulmonary disease (COPD) across a wide range of disease severity. The results showed that the proposed co-segmentation approach yielded superior performance compared to single-channel image segmentation in terms of precision, accuracy and robustness.

Ultrasound guidance for beating heart mitral valve repair augmented by synthetic dynamic CT.

Minimally invasive valvular intervention commonly requires intra-procedural navigation to provide spatial and temporal information of relevant cardiac structures and device components. Recently intra-procedural trans-esophageal echocardiography (TEE) has been exploited for this purpose due to its accessibility, low cost, ease of use, and real-time imaging capacity. However, the position and orientation of tissue targets relative to surgical tools can be challenging to perceive, particularly using 2D imaging planes. In this paper, we propose the use of CT images to provide a high-quality 3D context to enhance ultrasound images through image registration, providing an augmented guidance system with minimal impact on standard clinical workflow. We also describe an approach to generate synthetic 4D CT images through non-rigid registration of available ultrasound. This can be employed to avoid a requirement for higher radiation. Synthetic CT images were validated through direct comparison of synthetic and real multi-phase CT images. Validation of CT and ultrasound image registration was performed for both dynamic and synthetic CT image datasets. Our results demonstrated that the synthetically generated dynamic CT images provide similar anatomical representation for relevant cardiac anatomy relative to real dynamic CT images, and similar high registration accuracy that can be achieved for intra-procedural TEE to this versus real dynamic CT images.