Accurate cortical tissue classification on MRI by modeling cortical folding patterns.

Accurate tissue classification is a crucial prerequisite to MRI morphometry. Automated methods based on intensity histograms constructed from the entire volume are challenged by regional intensity variations due to local radiofrequency artifacts as well as disparities in tissue composition, laminar architecture and folding patterns. Current work proposes a novel anatomy-driven method in which parcels conforming cortical folding were regionally extracted from the brain. Each parcel is subsequently classified using nonparametric mean shift clustering. Evaluation was carried out on manually labeled images from two datasets acquired at 3.0 Tesla (n = 15) and 1.5 Tesla (n = 20). In both datasets, we observed high tissue classification accuracy of the proposed method (Dice index >97.6% at 3.0 Tesla, and >89.2% at 1.5 Tesla). Moreover, our method consistently outperformed state-of-the-art classification routines available in SPM8 and FSL-FAST, as well as a recently proposed local classifier that partitions the brain into cubes. Contour-based analyses localized more accurate white matter-gray matter (GM) interface classification of the proposed framework compared to the other algorithms, particularly in central and occipital cortices that generally display bright GM due to their highly degree of myelination. Excellent accuracy was maintained, even in the absence of correction for intensity inhomogeneity. The presented anatomy-driven local classification algorithm may significantly improve cortical boundary definition, with possible benefits for morphometric inference and biomarker discovery.

Development and deployment of a histopathology-based deep learning algorithm for patient prescreening in a clinical trial.

Accurate identification of genetic alterations in tumors, such as Fibroblast Growth Factor Receptor, is crucial for treating with targeted therapies; however, molecular testing can delay patient care due to the time and tissue required. Successful development, validation, and deployment of an AI-based, biomarker-detection algorithm could reduce screening cost and accelerate patient recruitment. Here, we develop a deep-learning algorithm using >3000 H&E-stained whole slide images from patients with advanced urothelial cancers, optimized for high sensitivity to avoid ruling out trial-eligible patients. The algorithm is validated on a dataset of 350 patients, achieving an area under the curve of 0.75, specificity of 31.8% at 88.7% sensitivity, and projected 28.7% reduction in molecular testing. We successfully deploy the system in a non-interventional study comprising 89 global study clinical sites and demonstrate its potential to prioritize/deprioritize molecular testing resources and provide substantial cost savings in the drug development and clinical settings.

A comparative study on computational models of multi-electrode radiofrequency ablation of large liver tumors.

PURPOSE: Thermal ablation of liver tumors has emerged as a first-line curative treatment for single small tumors (diameter < 2.5 cm) due to similar overall survival rates as surgical resection. Moreover, it is far less invasive, has lower complication rates, a superior cost-effectiveness, and an extremely low treatment-associated mortality. However, in many cases, complete tumor coverage cannot be achieved only with a single electrode and several electrodes are used to create overlapping ablations. Multi-electrode planning is a challenging 3D task with many contradictive constraints to consider, a dimensionality difficult to assess even for experts. It requires extremely long planning time since it is mostly performed mentally by clinicians looking at 2D CT views. An accurate and reliable prediction of the ablation zone would help to turn thermal ablation into a first-line curative treatment also for large liver tumors treated with multiple electrodes. In order to determine the level of model simplification that can be acceptable, we compared three computational models, a simple spherical model, a biophysics-based model and an Eikonal model. METHODS: RF ablation electrodes were virtually placed at a desired position in the patient pre-operative CT image and the models predicted the ablation zone generated by multiple electrodes. The last two models are patient-specific. In these cases, hepatic structures were automatically segmented from the pre-operative CT images to predict a patient-specific ablation zone. RESULTS: The three models were used to simulate multiple electrode ablations on 12 large tumors from 11 patients for which the procedure information was available. Biophysics-based simulations approximate better the post-operative ablation zone in term of Hausdorff distance, Dice Similarity Coefficient, radius, and volume compared to two other methods. It also predicts better the coverage percentage and thus the tumor ablation margin. CONCLUSION: The results obtained with the biophysics-based model indicate that it could improve ablation planning by accurately predicting the ablation zone, avoiding over or under-treatment. This is particularly beneficial for multi-electrode radiofrequency ablation of larger liver tumors where the planning phase is particularly challenging.

Automated catheter tip repositioning for intra-cardiac echocardiography.

PURPOSE: Intra-Cardiac Echocardiography (ICE) is a powerful imaging modality for guiding cardiac electrophysiology and structural heart interventions. ICE provides real-time observation of anatomy and devices, while enabling direct monitoring of potential complications. In single operator settings, the physician needs to switch back-and-forth between the ICE catheter and therapy device, making continuous ICE support impossible. Two operator setups are sometimes implemented, but increase procedural costs and room occupation. METHODS: ICE catheter robotic control system is developed with automated catheter tip repositioning (i.e., view recovery) method, which can reproduce important views previously navigated to and saved by the user. The performance of the proposed method is demonstrated and evaluated in a combination of heart phantom and animal experiments. RESULTS: Automated ICE view recovery achieved catheter tip position accuracy of [Formula: see text] mm and catheter image orientation accuracy of [Formula: see text] in animal studies, and [Formula: see text]mm and [Formula: see text] in heart phantom studies, respectively. Our proposed method is also successfully used during transseptal puncture in animals without complications, showing the possibility for fluoro-less transseptal puncture with ICE catheter robot. CONCLUSION: Robotic ICE imaging has the potential to provide precise and reproducible anatomical views, which can reduce overall execution time, labor burden of procedures, and X-ray usage for a range of cardiac procedures.

Learning a Generative Motion Model From Image Sequences Based on a Latent Motion Matrix.

We propose to learn a probabilistic motion model from a sequence of images for spatio-temporal registration. Our model encodes motion in a low-dimensional probabilistic space - the motion matrix - which enables various motion analysis tasks such as simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation. More precisely, the motion matrix allows to transport the recovered motion from one subject to another simulating for example a pathological motion in a healthy subject without the need for inter-subject registration. The method is based on a conditional latent variable model that is trained using amortized variational inference. This unsupervised generative model follows a novel multivariate Gaussian process prior and is applied within a temporal convolutional network which leads to a diffeomorphic motion model. Temporal consistency and generalizability is further improved by applying a temporal dropout training scheme. Applied to cardiac cine-MRI sequences, we show improved registration accuracy and spatio-temporally smoother deformations compared to three state-of-the-art registration algorithms. Besides, we demonstrate the model's applicability for motion analysis, simulation and super-resolution by an improved motion reconstruction from sequences with missing frames compared to linear and cubic interpolation.

Extrapolation of Ventricular Activation Times From Sparse Electroanatomical Data Using Graph Convolutional Neural Networks.

Electroanatomic mapping is the gold standard for the assessment of ventricular tachycardia. Acquiring high resolution electroanatomic maps is technically challenging and may require interpolation methods to obtain dense measurements. These methods, however, cannot recover activation times in the entire biventricular domain. This work investigates the use of graph convolutional neural networks to estimate biventricular activation times from sparse measurements. Our method is trained on more than 15,000 synthetic examples of realistic ventricular depolarization patterns generated by a computational electrophysiology model. Using geometries sampled from a statistical shape model of biventricular anatomy, diverse wave dynamics are induced by randomly sampling scar and border zone distributions, locations of initial activation, and tissue conduction velocities. Once trained, the method accurately reconstructs biventricular activation times in left-out synthetic simulations with a mean absolute error of 3.9 ms ± 4.2 ms at a sampling density of one measurement sample per cm2. The total activation time is matched with a mean error of 1.4 ms ± 1.4 ms. A significant decrease in errors is observed in all heart zones with an increased number of samples. Without re-training, the network is further evaluated on two datasets: (1) an in-house dataset comprising four ischemic porcine hearts with dense endocardial activation maps; (2) the CRT-EPIGGY19 challenge data comprising endo- and epicardial measurements of 5 infarcted and 6 non-infarcted swines. In both setups the neural network recovers biventricular activation times with a mean absolute error of less than 10 ms even when providing only a subset of endocardial measurements as input. Furthermore, we present a simple approach to suggest new measurement locations in real-time based on the estimated uncertainty of the graph network predictions. The model-guided selection of measurement locations allows to reduce by 40% the number of measurements required in a random sampling strategy, while achieving the same prediction error. In all the tested scenarios, the proposed approach estimates biventricular activation times with comparable or better performance than a personalized computational model and significant runtime advantages.

The Way of the Future: Personalizing Treatment Plans Through Technology.

Advances in tissue analysis methods, image analysis, high-throughput molecular profiling, and computational tools increasingly allow us to capture and quantify patient-to patient variations that impact cancer risk, prognosis, and treatment response. Statistical models that integrate patient-specific information from multiple sources (e.g., family history, demographics, germline variants, imaging features) can provide individualized cancer risk predictions that can guide screening and prevention strategies. The precision, quality, and standardization of diagnostic imaging are improving through computer-aided solutions, and multigene prognostic and predictive tests improved predictions of prognosis and treatment response in various cancer types. A common theme across many of these advances is that individually moderately informative variables are combined into more accurate multivariable prediction models. Advances in machine learning and the availability of large data sets fuel rapid progress in this field. Molecular dissection of the cancer genome has become a reality in the clinic, and molecular target profiling is now routinely used to select patients for various targeted therapies. These technology-driven increasingly more precise and quantitative estimates of benefit versus risk from a given intervention empower patients and physicians to tailor treatment strategies that match patient values and expectations.

CinE caRdiac magneTic resonAnce to predIct veNTricular arrhYthmia (CERTAINTY).

Better models to identify individuals at low risk of ventricular arrhythmia (VA) are needed for implantable cardioverter-defibrillator (ICD) candidates to mitigate the risk of ICD-related complications. We designed the CERTAINTY study (CinE caRdiac magneTic resonAnce to predIct veNTricular arrhYthmia) with deep learning for VA risk prediction from cine cardiac magnetic resonance (CMR). Using a training cohort of primary prevention ICD recipients (n = 350, 97 women, median age 59 years, 178 ischemic cardiomyopathy) who underwent CMR immediately prior to ICD implantation, we developed two neural networks: Cine Fingerprint Extractor and Risk Predictor. The former extracts cardiac structure and function features from cine CMR in a form of cine fingerprint in a fully unsupervised fashion, and the latter takes in the cine fingerprint and outputs disease outcomes as a cine risk score. Patients with VA (n = 96) had a significantly higher cine risk score than those without VA. Multivariate analysis showed that the cine risk score was significantly associated with VA after adjusting for clinical characteristics, cardiac structure and function including CMR-derived scar extent. These findings indicate that non-contrast, cine CMR inherently contains features to improve VA risk prediction in primary prevention ICD candidates. We solicit participation from multiple centers for external validation.

Unsupervised X-ray image segmentation with task driven generative adversarial networks.

Semantic parsing of anatomical structures in X-ray images is a critical task in many clinical applications. Modern methods leverage deep convolutional networks, and generally require a large amount of labeled data for model training. However, obtaining accurate pixel-wise labels on X-ray images is very challenging due to the appearance of anatomy overlaps and complex texture patterns. In comparison, labeled CT data are more accessible since organs in 3D CT scans preserve clearer structures and thus can be easily delineated. In this paper, we propose a model framework for learning automatic X-ray image parsing from labeled 3D CT scans. Specifically, a Deep Image-to-Image network (DI2I) for multi-organ segmentation is first trained on X-ray like Digitally Reconstructed Radiographs (DRRs) rendered from 3D CT volumes. Then we build a Task Driven Generative Adversarial Network (TD-GAN) to achieve simultaneous synthesis and parsing for unseen real X-ray images. The entire model pipeline does not require any annotations from the X-ray image domain. In the numerical experiments, we validate the proposed model on over 800 DRRs and 300 topograms. While the vanilla DI2I trained on DRRs without any adaptation fails completely on segmenting the topograms, the proposed model does not require any topogram labels and is able to provide a promising average dice of 86% which achieves the same level of accuracy as results from supervised training (89%). Furthermore, we also demonstrate the generality of TD-GAN through quantatitive and qualitative study on widely used public dataset.

Learning a Probabilistic Model for Diffeomorphic Registration.

We propose to learn a low-dimensional probabilistic deformation model from data which can be used for the registration and the analysis of deformations. The latent variable model maps similar deformations close to each other in an encoding space. It enables to compare deformations, to generate normal or pathological deformations for any new image, or to transport deformations from one image pair to any other image. Our unsupervised method is based on the variational inference. In particular, we use a conditional variational autoencoder network and constrain transformations to be symmetric and diffeomorphic by applying a differentiable exponentiation layer with a symmetric loss function. We also present a formulation that includes spatial regularization such as the diffusion-based filters. In addition, our framework provides multi-scale velocity field estimations. We evaluated our method on 3-D intra-subject registration using 334 cardiac cine-MRIs. On this dataset, our method showed the state-of-the-art performance with a mean DICE score of 81.2% and a mean Hausdorff distance of 7.3 mm using 32 latent dimensions compared to three state-of-the-art methods while also demonstrating more regular deformation fields. The average time per registration was 0.32 s. Besides, we visualized the learned latent space and showed that the encoded deformations can be used to transport deformations and to cluster diseases with a classification accuracy of 83% after applying a linear projection.

Toward patient-specific myocardial models of the heart.

This article presents a framework for building patient-specific models of the myocardium, to help diagnosis, therapy planning, and procedure guidance. The aim is to be able to introduce such models in clinical applications. Thus, there is a need to design models that can be adjusted from clinical data, images, or signals, which are sparse and noisy. The authors describe the three main components of a myocardial model: the anatomy, the electrophysiology, and the biomechanics. For each of these components, the authors try to obtain the best balance between prior knowledge and observable parameters to be able to adjust these models to patient data. To achieve this, there is a need to design models with the right level of complexity and a computational cost compatible with clinical constraints.

3D/2D model-to-image registration by imitation learning for cardiac procedures.

PURPOSE: In cardiac interventions, such as cardiac resynchronization therapy (CRT), image guidance can be enhanced by involving preoperative models. Multimodality 3D/2D registration for image guidance, however, remains a significant research challenge for fundamentally different image data, i.e., MR to X-ray. Registration methods must account for differences in intensity, contrast levels, resolution, dimensionality, field of view. Furthermore, same anatomical structures may not be visible in both modalities. Current approaches have focused on developing modality-specific solutions for individual clinical use cases, by introducing constraints, or identifying cross-modality information manually. Machine learning approaches have the potential to create more general registration platforms. However, training image to image methods would require large multimodal datasets and ground truth for each target application. METHODS: This paper proposes a model-to-image registration approach instead, because it is common in image-guided interventions to create anatomical models for diagnosis, planning or guidance prior to procedures. An imitation learning-based method, trained on 702 datasets, is used to register preoperative models to intraoperative X-ray images. RESULTS: Accuracy is demonstrated on cardiac models and artificial X-rays generated from CTs. The registration error was [Formula: see text] on 1000 test cases, superior to that of manual ([Formula: see text]) and gradient-based ([Formula: see text]) registration. High robustness is shown in 19 clinical CRT cases. CONCLUSION: Besides the proposed methods feasibility in a clinical environment, evaluation has shown good accuracy and high robustness indicating that it could be applied in image-guided interventions.

Comprehensive preclinical evaluation of a multi-physics model of liver tumor radiofrequency ablation.

PURPOSE: We aim at developing a framework for the validation of a subject-specific multi-physics model of liver tumor radiofrequency ablation (RFA). METHODS: The RFA computation becomes subject specific after several levels of personalization: geometrical and biophysical (hemodynamics, heat transfer and an extended cellular necrosis model). We present a comprehensive experimental setup combining multimodal, pre- and postoperative anatomical and functional images, as well as the interventional monitoring of intra-operative signals: the temperature and delivered power. RESULTS: To exploit this dataset, an efficient processing pipeline is introduced, which copes with image noise, variable resolution and anisotropy. The validation study includes twelve ablations from five healthy pig livers: a mean point-to-mesh error between predicted and actual ablation extent of 5.3 ± 3.6 mm is achieved. CONCLUSION: This enables an end-to-end preclinical validation framework that considers the available dataset.

Towards patient-specific modeling of mitral valve repair: 3D transesophageal echocardiography-derived parameter estimation.

Transesophageal echocardiography (TEE) is routinely used to provide important qualitative and quantitative information regarding mitral regurgitation. Contemporary planning of surgical mitral valve repair, however, still relies heavily upon subjective predictions based on experience and intuition. While patient-specific mitral valve modeling holds promise, its effectiveness is limited by assumptions that must be made about constitutive material properties. In this paper, we propose and develop a semi-automated framework that combines machine learning image analysis with geometrical and biomechanical models to build a patient-specific mitral valve representation that incorporates image-derived material properties. We use our computational framework, along with 3D TEE images of the open and closed mitral valve, to estimate values for chordae rest lengths and leaflet material properties. These parameters are initialized using generic values and optimized to match the visualized deformation of mitral valve geometry between the open and closed states. Optimization is achieved by minimizing the summed Euclidean distances between the estimated and image-derived closed mitral valve geometry. The spatially varying material parameters of the mitral leaflets are estimated using an extended Kalman filter to take advantage of the temporal information available from TEE. This semi-automated and patient-specific modeling framework was tested on 15 TEE image acquisitions from 14 patients. Simulated mitral valve closures yielded average errors (measured by point-to-point Euclidean distances) of 1.86 ± 1.24 mm. The estimated material parameters suggest that the anterior leaflet is stiffer than the posterior leaflet and that these properties vary between individuals, consistent with experimental observations described in the literature.

Personalized mitral valve closure computation and uncertainty analysis from 3D echocardiography.

Intervention planning is essential for successful Mitral Valve (MV) repair procedures. Finite-element models (FEM) of the MV could be used to achieve this goal, but the translation to the clinical domain is challenging. Many input parameters for the FEM models, such as tissue properties, are not known. In addition, only simplified MV geometry models can be extracted from non-invasive modalities such as echocardiography imaging, lacking major anatomical details such as the complex chordae topology. A traditional approach for FEM computation is to use a simplified model (also known as parachute model) of the chordae topology, which connects the papillary muscle tips to the free-edges and select basal points. Building on the existing parachute model a new and comprehensive MV model was developed that utilizes a novel chordae representation capable of approximating regional connectivity. In addition, a fully automated personalization approach was developed for the chordae rest length, removing the need for tedious manual parameter selection. Based on the MV model extracted during mid-diastole (open MV) the MV geometric configuration at peak systole (closed MV) was computed according to the FEM model. In this work the focus was placed on validating MV closure computation. The method is evaluated on ten in vitro ovine cases, where in addition to echocardiography imaging, high-resolution µCT imaging is available for accurate validation.

Shaping the future through innovations: From medical imaging to precision medicine.

Medical images constitute a source of information essential for disease diagnosis, treatment and follow-up. In addition, due to its patient-specific nature, imaging information represents a critical component required for advancing precision medicine into clinical practice. This manuscript describes recently developed technologies for better handling of image information: photorealistic visualization of medical images with Cinematic Rendering, artificial agents for in-depth image understanding, support for minimally invasive procedures, and patient-specific computational models with enhanced predictive power. Throughout the manuscript we will analyze the capabilities of such technologies and extrapolate on their potential impact to advance the quality of medical care, while reducing its cost.

A self-taught artificial agent for multi-physics computational model personalization.

Personalization is the process of fitting a model to patient data, a critical step towards application of multi-physics computational models in clinical practice. Designing robust personalization algorithms is often a tedious, time-consuming, model- and data-specific process. We propose to use artificial intelligence concepts to learn this task, inspired by how human experts manually perform it. The problem is reformulated in terms of reinforcement learning. In an off-line phase, Vito, our self-taught artificial agent, learns a representative decision process model through exploration of the computational model: it learns how the model behaves under change of parameters. The agent then automatically learns an optimal strategy for on-line personalization. The algorithm is model-independent; applying it to a new model requires only adjusting few hyper-parameters of the agent and defining the observations to match. The full knowledge of the model itself is not required. Vito was tested in a synthetic scenario, showing that it could learn how to optimize cost functions generically. Then Vito was applied to the inverse problem of cardiac electrophysiology and the personalization of a whole-body circulation model. The obtained results suggested that Vito could achieve equivalent, if not better goodness of fit than standard methods, while being more robust (up to 11% higher success rates) and with faster (up to seven times) convergence rate. Our artificial intelligence approach could thus make personalization algorithms generalizable and self-adaptable to any patient and any model.

Personalized Computer Simulation of Diastolic Function in Heart Failure.

The search for a parameter representing left ventricular relaxation from non-invasive and invasive diagnostic tools has been extensive, since heart failure (HF) with preserved ejection fraction (HF-pEF) is a global health problem. We explore here the feasibility using patient-specific cardiac computer modeling to capture diastolic parameters in patients suffering from different degrees of systolic HF. Fifty eight patients with idiopathic dilated cardiomyopathy have undergone thorough clinical evaluation, including cardiac magnetic resonance imaging (MRI), heart catheterization, echocardiography, and cardiac biomarker assessment. A previously-introduced framework for creating multi-scale patient-specific cardiac models has been applied on all these patients. Novel parameters, such as global stiffness factor and maximum left ventricular active stress, representing cardiac active and passive tissue properties have been computed for all patients. Invasive pressure measurements from heart catheterization were then used to evaluate ventricular relaxation using the time constant of isovolumic relaxation Tau (τ). Parameters from heart catheterization and the multi-scale model have been evaluated and compared to patient clinical presentation. The model parameter global stiffness factor, representing diastolic passive tissue properties, is correlated significantly across the patient population with τ. This study shows that multi-modal cardiac models can successfully capture diastolic (dys) function, a prerequisite for future clinical trials on HF-pEF.

Multi-contrast submillimetric 3 Tesla hippocampal subfield segmentation protocol and dataset.

The hippocampus is composed of distinct anatomical subregions that participate in multiple cognitive processes and are differentially affected in prevalent neurological and psychiatric conditions. Advances in high-field MRI allow for the non-invasive identification of hippocampal substructure. These approaches, however, demand time-consuming manual segmentation that relies heavily on anatomical expertise. Here, we share manual labels and associated high-resolution MRI data (MNI-HISUB25; submillimetric T1- and T2-weighted images, detailed sequence information, and stereotaxic probabilistic anatomical maps) based on 25 healthy subjects. Data were acquired on a widely available 3 Tesla MRI system using a 32 phased-array head coil. The protocol divided the hippocampal formation into three subregions: subicular complex, merged Cornu Ammonis 1, 2 and 3 (CA1-3) subfields, and CA4-dentate gyrus (CA4-DG). Segmentation was guided by consistent intensity and morphology characteristics of the densely myelinated molecular layer together with few geometry-based boundaries flexible to overall mesiotemporal anatomy, and achieved excellent intra-/inter-rater reliability (Dice index ≥90/87%). The dataset can inform neuroimaging assessments of the mesiotemporal lobe and help to develop segmentation algorithms relevant for basic and clinical neurosciences.