Development and deployment of a histopathology-based deep learning algorithm for patient prescreening in a clinical trial.

Accurate identification of genetic alterations in tumors, such as Fibroblast Growth Factor Receptor, is crucial for treating with targeted therapies; however, molecular testing can delay patient care due to the time and tissue required. Successful development, validation, and deployment of an AI-based, biomarker-detection algorithm could reduce screening cost and accelerate patient recruitment. Here, we develop a deep-learning algorithm using >3000 H&E-stained whole slide images from patients with advanced urothelial cancers, optimized for high sensitivity to avoid ruling out trial-eligible patients. The algorithm is validated on a dataset of 350 patients, achieving an area under the curve of 0.75, specificity of 31.8% at 88.7% sensitivity, and projected 28.7% reduction in molecular testing. We successfully deploy the system in a non-interventional study comprising 89 global study clinical sites and demonstrate its potential to prioritize/deprioritize molecular testing resources and provide substantial cost savings in the drug development and clinical settings.

A comparative study on computational models of multi-electrode radiofrequency ablation of large liver tumors.

PURPOSE: Thermal ablation of liver tumors has emerged as a first-line curative treatment for single small tumors (diameter < 2.5 cm) due to similar overall survival rates as surgical resection. Moreover, it is far less invasive, has lower complication rates, a superior cost-effectiveness, and an extremely low treatment-associated mortality. However, in many cases, complete tumor coverage cannot be achieved only with a single electrode and several electrodes are used to create overlapping ablations. Multi-electrode planning is a challenging 3D task with many contradictive constraints to consider, a dimensionality difficult to assess even for experts. It requires extremely long planning time since it is mostly performed mentally by clinicians looking at 2D CT views. An accurate and reliable prediction of the ablation zone would help to turn thermal ablation into a first-line curative treatment also for large liver tumors treated with multiple electrodes. In order to determine the level of model simplification that can be acceptable, we compared three computational models, a simple spherical model, a biophysics-based model and an Eikonal model. METHODS: RF ablation electrodes were virtually placed at a desired position in the patient pre-operative CT image and the models predicted the ablation zone generated by multiple electrodes. The last two models are patient-specific. In these cases, hepatic structures were automatically segmented from the pre-operative CT images to predict a patient-specific ablation zone. RESULTS: The three models were used to simulate multiple electrode ablations on 12 large tumors from 11 patients for which the procedure information was available. Biophysics-based simulations approximate better the post-operative ablation zone in term of Hausdorff distance, Dice Similarity Coefficient, radius, and volume compared to two other methods. It also predicts better the coverage percentage and thus the tumor ablation margin. CONCLUSION: The results obtained with the biophysics-based model indicate that it could improve ablation planning by accurately predicting the ablation zone, avoiding over or under-treatment. This is particularly beneficial for multi-electrode radiofrequency ablation of larger liver tumors where the planning phase is particularly challenging.

Automated catheter tip repositioning for intra-cardiac echocardiography.

PURPOSE: Intra-Cardiac Echocardiography (ICE) is a powerful imaging modality for guiding cardiac electrophysiology and structural heart interventions. ICE provides real-time observation of anatomy and devices, while enabling direct monitoring of potential complications. In single operator settings, the physician needs to switch back-and-forth between the ICE catheter and therapy device, making continuous ICE support impossible. Two operator setups are sometimes implemented, but increase procedural costs and room occupation. METHODS: ICE catheter robotic control system is developed with automated catheter tip repositioning (i.e., view recovery) method, which can reproduce important views previously navigated to and saved by the user. The performance of the proposed method is demonstrated and evaluated in a combination of heart phantom and animal experiments. RESULTS: Automated ICE view recovery achieved catheter tip position accuracy of [Formula: see text] mm and catheter image orientation accuracy of [Formula: see text] in animal studies, and [Formula: see text]mm and [Formula: see text] in heart phantom studies, respectively. Our proposed method is also successfully used during transseptal puncture in animals without complications, showing the possibility for fluoro-less transseptal puncture with ICE catheter robot. CONCLUSION: Robotic ICE imaging has the potential to provide precise and reproducible anatomical views, which can reduce overall execution time, labor burden of procedures, and X-ray usage for a range of cardiac procedures.

The Way of the Future: Personalizing Treatment Plans Through Technology.

Advances in tissue analysis methods, image analysis, high-throughput molecular profiling, and computational tools increasingly allow us to capture and quantify patient-to patient variations that impact cancer risk, prognosis, and treatment response. Statistical models that integrate patient-specific information from multiple sources (e.g., family history, demographics, germline variants, imaging features) can provide individualized cancer risk predictions that can guide screening and prevention strategies. The precision, quality, and standardization of diagnostic imaging are improving through computer-aided solutions, and multigene prognostic and predictive tests improved predictions of prognosis and treatment response in various cancer types. A common theme across many of these advances is that individually moderately informative variables are combined into more accurate multivariable prediction models. Advances in machine learning and the availability of large data sets fuel rapid progress in this field. Molecular dissection of the cancer genome has become a reality in the clinic, and molecular target profiling is now routinely used to select patients for various targeted therapies. These technology-driven increasingly more precise and quantitative estimates of benefit versus risk from a given intervention empower patients and physicians to tailor treatment strategies that match patient values and expectations.

3D/2D model-to-image registration by imitation learning for cardiac procedures.

PURPOSE: In cardiac interventions, such as cardiac resynchronization therapy (CRT), image guidance can be enhanced by involving preoperative models. Multimodality 3D/2D registration for image guidance, however, remains a significant research challenge for fundamentally different image data, i.e., MR to X-ray. Registration methods must account for differences in intensity, contrast levels, resolution, dimensionality, field of view. Furthermore, same anatomical structures may not be visible in both modalities. Current approaches have focused on developing modality-specific solutions for individual clinical use cases, by introducing constraints, or identifying cross-modality information manually. Machine learning approaches have the potential to create more general registration platforms. However, training image to image methods would require large multimodal datasets and ground truth for each target application. METHODS: This paper proposes a model-to-image registration approach instead, because it is common in image-guided interventions to create anatomical models for diagnosis, planning or guidance prior to procedures. An imitation learning-based method, trained on 702 datasets, is used to register preoperative models to intraoperative X-ray images. RESULTS: Accuracy is demonstrated on cardiac models and artificial X-rays generated from CTs. The registration error was [Formula: see text] on 1000 test cases, superior to that of manual ([Formula: see text]) and gradient-based ([Formula: see text]) registration. High robustness is shown in 19 clinical CRT cases. CONCLUSION: Besides the proposed methods feasibility in a clinical environment, evaluation has shown good accuracy and high robustness indicating that it could be applied in image-guided interventions.

Comprehensive preclinical evaluation of a multi-physics model of liver tumor radiofrequency ablation.

PURPOSE: We aim at developing a framework for the validation of a subject-specific multi-physics model of liver tumor radiofrequency ablation (RFA). METHODS: The RFA computation becomes subject specific after several levels of personalization: geometrical and biophysical (hemodynamics, heat transfer and an extended cellular necrosis model). We present a comprehensive experimental setup combining multimodal, pre- and postoperative anatomical and functional images, as well as the interventional monitoring of intra-operative signals: the temperature and delivered power. RESULTS: To exploit this dataset, an efficient processing pipeline is introduced, which copes with image noise, variable resolution and anisotropy. The validation study includes twelve ablations from five healthy pig livers: a mean point-to-mesh error between predicted and actual ablation extent of 5.3 ± 3.6 mm is achieved. CONCLUSION: This enables an end-to-end preclinical validation framework that considers the available dataset.

Shaping the future through innovations: From medical imaging to precision medicine.

Medical images constitute a source of information essential for disease diagnosis, treatment and follow-up. In addition, due to its patient-specific nature, imaging information represents a critical component required for advancing precision medicine into clinical practice. This manuscript describes recently developed technologies for better handling of image information: photorealistic visualization of medical images with Cinematic Rendering, artificial agents for in-depth image understanding, support for minimally invasive procedures, and patient-specific computational models with enhanced predictive power. Throughout the manuscript we will analyze the capabilities of such technologies and extrapolate on their potential impact to advance the quality of medical care, while reducing its cost.

Efficient Lattice Boltzmann Solver for Patient-Specific Radiofrequency Ablation of Hepatic Tumors.

Radiofrequency ablation (RFA) is an established treatment for liver cancer when resection is not possible. Yet, its optimal delivery is challenged by the presence of large blood vessels and the time-varying thermal conductivity of biological tissue. Incomplete treatment and an increased risk of recurrence are therefore common. A tool that would enable the accurate planning of RFA is hence necessary. This manuscript describes a new method to compute the extent of ablation required based on the Lattice Boltzmann Method (LBM) and patient-specific, pre-operative images. A detailed anatomical model of the liver is obtained from volumetric images. Then a computational model of heat diffusion, cellular necrosis, and blood flow through the vessels and liver is employed to compute the extent of ablated tissue given the probe location, ablation duration and biological parameters. The model was verified against an analytical solution, showing good fidelity. We also evaluated the predictive power of the proposed framework on ten patients who underwent RFA, for whom pre- and post-operative images were available. Comparisons between the computed ablation extent and ground truth, as observed in postoperative images, were promising (DICE index: 42%, sensitivity: 67%, positive predictive value: 38%). The importance of considering liver perfusion while simulating electrical-heating ablation was also highlighted. Implemented on graphics processing units (GPU), our method simulates 1 minute of ablation in 1.14 minutes, allowing near real-time computation.

Disentangling hippocampal shape anomalies in epilepsy.

Drug-resistant temporal lobe epilepsy (TLE) and epileptic syndromes related to malformations of cortical development (MCD) are associated with complex hippocampal morphology. The contribution of volume and position to the overall hippocampal shape in these conditions has not been studied. We propose a surface-based framework to localize volume changes through measurement of Jacobian determinants, and quantify fine-scale position and curvature through a medial axis model. We applied our methodology to T1-weighted 3D volumetric MRI of 88 patients with TLE and 78 patients with MCD, including focal cortical dysplasia (FCD, n = 29), heterotopia (HET, n = 40), and polymicrogyria (PMG, n = 19). Patients were compared to 46 age- and sex-matched healthy controls. Surface-based analysis of volume in TLE revealed severe ipsilateral atrophy mainly along the rostro-caudal extent of the hippocampal CA1 subfield. In MCD, patterns of volume changes included bilateral CA1 atrophy in HET and FCD, and left dentate hypertrophy in all three groups. The analysis of curvature revealed medial bending of the posterior hippocampus in TLE, whereas in MCD there was a supero-medial shift of the hippocampal body. Albeit hippocampal shape anomalies in TLE and MCD result from a combination of volume and positional changes, their nature and distribution suggest different pathogenic mechanisms.

Biomechanically driven registration of pre- to intra-operative 3D images for laparoscopic surgery.

Minimally invasive laparoscopic surgery is widely used for the treatment of cancer and other diseases. During the procedure, gas insufflation is used to create space for laparoscopic tools and operation. Insufflation causes the organs and abdominal wall to deform significantly. Due to this large deformation, the benefit of surgical plans, which are typically based on pre-operative images, is limited for real time navigation. In some recent work, intra-operative images, such as cone-beam CT or interventional CT, are introduced to provide updated volumetric information after insufflation. Other works in this area have focused on simulation of gas insufflation and exploited only the pre-operative images to estimate deformation. This paper proposes a novel registration method for pre- and intra-operative 3D image fusion for laparoscopic surgery. In this approach, the deformation of pre-operative images is driven by a biomechanical model of the insufflation process. The proposed method was validated by five synthetic data sets generated from clinical images and three pairs of in vivo CT scans acquired from two pigs, before and after insufflation. The results show the proposed method achieved high accuracy for both the synthetic and real insufflation data.

Image-based computational models for TAVI planning: from CT images to implant deployment.

Transcatheter aortic valve implantation (TAVI) is becoming the standard choice of care for non-operable patients suffering from severe aortic valve stenosis. As there is no direct view or access to the affected anatomy, accurate preoperative planning is crucial for a successful outcome. The most important decision during planning is selecting the proper implant type and size. Due to the wide variety in device sizes and types and non-circular annulus shapes, there is often no obvious choice for the specific patient. Most clinicians base their final decision on their previous experience. As a first step towards a more predictive planning, we propose an integrated method to estimate the aortic apparatus from CT images and compute implant deployment. Aortic anatomy, which includes aortic root, leaflets and calcifications, is automatically extracted using robust modeling and machine learning algorithms. Then, the finite element method is employed to calculate the deployment of a TAVI implant inside the patient-specific aortic anatomy. The anatomical model was evaluated on 198 CT images, yielding an accuracy of 1.30 +/- 0.23 mm. In eleven subjects, pre- and post-TAVI CT images were available. Errors in predicted implant deployment were of 1.74 +/- 0.40 mm in average and 1.32 mm in the aortic valve annulus region, which is almost three times lower than the average gap of 3 mm between consecutive implant sizes. Our framework may thus constitute a surrogate tool for TAVI planning.

Lattice Boltzmann method for fast patient-specific simulation of liver tumor ablation from CT images.

Radio-frequency ablation (RFA), the most widely used minimally invasive ablative therapy of liver cancer, is challenged by a lack of patient-specific planning. In particular, the presence of blood vessels and time-varying thermal diffusivity makes the prediction of the extent of the ablated tissue difficult. This may result in incomplete treatments and increased risk of recurrence. We propose a new model of the physical mechanisms involved in RFA of abdominal tumors based on Lattice Boltzmann Method to predict the extent of ablation given the probe location and the biological parameters. Our method relies on patient images, from which level set representations of liver geometry, tumor shape and vessels are extracted. Then a computational model of heat diffusion, cellular necrosis and blood flow through vessels and liver is solved to estimate the extent of ablated tissue. After quantitative verifications against an analytical solution, we apply our framework to 5 patients datasets which include pre- and post-operative CT images, yielding promising correlation between predicted and actual ablation extent (mean point to mesh errors of 8.7 mm). Implemented on graphics processing units, our method may enable RFA planning in clinical settings as it leads to near real-time computation: 1 minute of ablation is simulated in 1.14 minutes, which is almost 60x faster than standard finite element method.

An integrated framework for finite-element modeling of mitral valve biomechanics from medical images: application to MitralClip intervention planning.

Treatment of mitral valve (MV) diseases requires comprehensive clinical evaluation and therapy personalization to optimize outcomes. Finite-element models (FEMs) of MV physiology have been proposed to study the biomechanical impact of MV repair, but their translation into the clinics remains challenging. As a step towards this goal, we present an integrated framework for finite-element modeling of the MV closure based on patient-specific anatomies and boundary conditions. Starting from temporal medical images, we estimate a comprehensive model of the MV apparatus dynamics, including papillary tips, using a machine-learning approach. A detailed model of the open MV at end-diastole is then computed, which is finally closed according to a FEM of MV biomechanics. The motion of the mitral annulus and papillary tips are constrained from the image data for increased accuracy. A sensitivity analysis of our system shows that chordae rest length and boundary conditions have a significant influence upon the simulation results. We quantitatively test the generalization of our framework on 25 consecutive patients. Comparisons between the simulated closed valve and ground truth show encouraging results (average point-to-mesh distance: 1.49 ± 0.62 mm) but also the need for personalization of tissue properties, as illustrated in three patients. Finally, the predictive power of our model is tested on one patient who underwent MitralClip by comparing the simulated intervention with the real outcome in terms of MV closure, yielding promising prediction. By providing an integrated way to perform MV simulation, our framework may constitute a surrogate tool for model validation and therapy planning.

Model-based fusion of multi-modal volumetric images: application to transcatheter valve procedures.

Minimal invasive procedures such as transcatheter valve interventions are substituting conventional surgical techniques. Thus, novel operating rooms have been designed to augment traditional surgical equipment with advanced imaging systems to guide the procedures. We propose a novel method to fuse pre-operative and intra-operative information by jointly estimating anatomical models from multiple image modalities. Thereby high-quality patient-specific models are integrated into the imaging environment of operating rooms to guide cardiac interventions. Robust and fast machine learning techniques are utilized to guide the estimation process. Our method integrates both the redundant and complementary multimodal information to achieve a comprehensive modeling and simultaneously reduce the estimation uncertainty. Experiments performed on 28 patients with pairs of multimodal volumetric data are used to demonstrate high quality intra-operative patient-specific modeling of the aortic valve with a precision of 1.09mm in TEE and 1.73mm in 3D C-arm CT. Within a processing time of 10 seconds we additionally obtain model sensitive mapping between the pre- and intraoperative images.

Robust surface-based multi-template automated algorithm to segment healthy and pathological hippocampi.

The most frequent drug-resistant epilepsy is temporal lobe epilepsy (TLE) related to hippocampal atrophy. In addition, TLE is associated with atypical hippocampal morphologies. Automatic hippocampal segmentations have generally provided unsatisfactory results in this condition. We propose a novel segmentation method (SurfMulti) to statistically estimate locoregional texture and shape using a surface-based approach that guarantees shape-inherent point-wise correspondences. To account for inter-subject variability, including shape variants, we used a multi-template library derived from a large database of controls and patients. SurfMulti outperformed state-of-the-art volume-based single- and multi-template approaches, with performances comparable to controls (Dice index: 86.1 vs. 87.5%). Furthermore, the sensitivity of SurfMulti to detect atrophy was similar to that of manual volumetry. Given that the presence of hippocampal atrophy in TLE predicts a favorable seizure outcome after surgery, the proposed automated algorithm assures to be a robust surrogate tool in the presurgical evaluation for the time-demanding manual procedure.