The STOIC2021 COVID-19 AI challenge: Applying reusable training methodologies to private data.

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.

Towards patient-specific modeling of mitral valve repair: 3D transesophageal echocardiography-derived parameter estimation.

Transesophageal echocardiography (TEE) is routinely used to provide important qualitative and quantitative information regarding mitral regurgitation. Contemporary planning of surgical mitral valve repair, however, still relies heavily upon subjective predictions based on experience and intuition. While patient-specific mitral valve modeling holds promise, its effectiveness is limited by assumptions that must be made about constitutive material properties. In this paper, we propose and develop a semi-automated framework that combines machine learning image analysis with geometrical and biomechanical models to build a patient-specific mitral valve representation that incorporates image-derived material properties. We use our computational framework, along with 3D TEE images of the open and closed mitral valve, to estimate values for chordae rest lengths and leaflet material properties. These parameters are initialized using generic values and optimized to match the visualized deformation of mitral valve geometry between the open and closed states. Optimization is achieved by minimizing the summed Euclidean distances between the estimated and image-derived closed mitral valve geometry. The spatially varying material parameters of the mitral leaflets are estimated using an extended Kalman filter to take advantage of the temporal information available from TEE. This semi-automated and patient-specific modeling framework was tested on 15 TEE image acquisitions from 14 patients. Simulated mitral valve closures yielded average errors (measured by point-to-point Euclidean distances) of 1.86 ± 1.24 mm. The estimated material parameters suggest that the anterior leaflet is stiffer than the posterior leaflet and that these properties vary between individuals, consistent with experimental observations described in the literature.

Noninvasive hemodynamic assessment, treatment outcome prediction and follow-up of aortic coarctation from MR imaging.

PURPOSE: Coarctation of the aorta (CoA) is a congenital heart disease characterized by an abnormal narrowing of the proximal descending aorta. Severity of this pathology is quantified by the blood pressure drop (△P) across the stenotic coarctation lesion. In order to evaluate the physiological significance of the preoperative coarctation and to assess the postoperative results, the hemodynamic analysis is routinely performed by measuring the △P across the coarctation site via invasive cardiac catheterization. The focus of this work is to present an alternative, noninvasive measurement of blood pressure drop △P through the introduction of a fast, image-based workflow for personalized computational modeling of the CoA hemodynamics. METHODS: The authors propose an end-to-end system comprised of shape and computational models, their personalization setup using MR imaging, and a fast, noninvasive method based on computational fluid dynamics (CFD) to estimate the pre- and postoperative hemodynamics for coarctation patients. A virtual treatment method is investigated to assess the predictive power of our approach. RESULTS: Automatic thoracic aorta segmentation was applied on a population of 212 3D MR volumes, with mean symmetric point-to-mesh error of 3.00 ± 1.58 mm and average computation time of 8 s. Through quantitative evaluation of 6 CoA patients, good agreement between computed blood pressure drop and catheter measurements is shown: average differences are 2.38 ± 0.82 mm Hg (pre-), 1.10 ± 0.63 mm Hg (postoperative), and 4.99 ± 3.00 mm Hg (virtual stenting), respectively. CONCLUSIONS: The complete workflow is realized in a fast, mostly-automated system that is integrable in the clinical setting. To the best of our knowledge, this is the first time that three different settings (preoperative--severity assessment, poststenting--follow-up, and virtual stenting--treatment outcome prediction) of CoA are investigated on multiple subjects. We believe that in future-given wider clinical validation-our noninvasive in-silico method could replace invasive pressure catheterization for CoA.

Probabilistic sparse matching for robust 3D/3D fusion in minimally invasive surgery.

Classical surgery is being overtaken by minimally invasive and transcatheter procedures. As there is no direct view or access to the affected anatomy, advanced imaging techniques such as 3D C-arm computed tomography (CT) and C-arm fluoroscopy are routinely used in clinical practice for intraoperative guidance. However, due to constraints regarding acquisition time and device configuration, intraoperative modalities have limited soft tissue image quality and reliable assessment of the cardiac anatomy typically requires contrast agent, which is harmful to the patient and requires complex acquisition protocols. We propose a probabilistic sparse matching approach to fuse high-quality preoperative CT images and nongated, noncontrast intraoperative C-arm CT images by utilizing robust machine learning and numerical optimization techniques. Thus, high-quality patient-specific models can be extracted from the preoperative CT and mapped to the intraoperative imaging environment to guide minimally invasive procedures. Extensive quantitative experiments on 95 clinical datasets demonstrate that our model-based fusion approach has an average execution time of 1.56 s, while the accuracy of 5.48 mm between the anchor anatomy in both images lies within expert user confidence intervals. In direct comparison with image-to-image registration based on an open-source state-of-the-art medical imaging library and a recently proposed quasi-global, knowledge-driven multi-modal fusion approach for thoracic-abdominal images, our model-based method exhibits superior performance in terms of registration accuracy and robustness with respect to both target anatomy and anchor anatomy alignment errors.

Four-dimensional modelling of the mitral valve by real-time 3D transoesophageal echocardiography: proof of concept.

OBJECTIVES: The complexity of the mitral valve (MV) anatomy and function is not yet fully understood. Assessing the dynamic movement and interaction of MV components to define MV physiology during the complete cardiac cycle remains a challenge. We herein describe a novel semi-automated 4D MV model. METHODS: The model applies quantitative analysis of the MV over a complete cardiac cycle based on real-time 3D transoesophageal echocardiography (RT3DE) data. RT3DE data of MVs were acquired for 18 patients. The MV annulus and leaflets were semi-automatically reconstructed. Dimensions of the mitral annulus (anteroposterior and anterolateral-posteromedial diameter, annular circumference, annular area) and leaflets (MV orifice area, intercommissural distance) were acquired. Variability and reproducibility (intraclass correlation coefficient, ICC) for interobserver and intraobserver comparison were quantified at 4 time points during the cardiac cycle (mid-systole, end-systole, mid-diastole and end-diastole). RESULTS: Mitral annular dimensions provided highly reliable and reproducible measurements throughout the cardiac cycle for interobserver (variability range, 0.5-1.5%; ICC range, 0.895-0.987) and intraobserver (variability range, 0.5-1.6%; ICC range, 0.827-0.980) comparison, respectively. MV leaflet parameters showed a high reliability in the diastolic phase (variability range, 0.6-9.1%; ICC range, 0.750-0.986), whereas MV leaflet dimensions showed a high variability and lower correlation in the systolic phase (variability range, 0.6-22.4%; ICC range, 0.446-0.915) compared with the diastolic phase. CONCLUSIONS: This 4D model provides detailed morphological reconstruction as well as sophisticated quantification of the complex MV structure and dynamics throughout the cardiac cycle with a precision not yet described.

Real-time ultrasound transducer localization in fluoroscopy images by transfer learning from synthetic training data.

The fusion of image data from trans-esophageal echography (TEE) and X-ray fluoroscopy is attracting increasing interest in minimally-invasive treatment of structural heart disease. In order to calculate the needed transformation between both imaging systems, we employ a discriminative learning (DL) based approach to localize the TEE transducer in X-ray images. The successful application of DL methods is strongly dependent on the available training data, which entails three challenges: (1) the transducer can move with six degrees of freedom meaning it requires a large number of images to represent its appearance, (2) manual labeling is time consuming, and (3) manual labeling has inherent errors. This paper proposes to generate the required training data automatically from a single volumetric image of the transducer. In order to adapt this system to real X-ray data, we use unlabeled fluoroscopy images to estimate differences in feature space density and correct covariate shift by instance weighting. Two approaches for instance weighting, probabilistic classification and Kullback-Leibler importance estimation (KLIEP), are evaluated for different stages of the proposed DL pipeline. An analysis on more than 1900 images reveals that our approach reduces detection failures from 7.3% in cross validation on the test set to zero and improves the localization error from 1.5 to 0.8mm. Due to the automatic generation of training data, the proposed system is highly flexible and can be adapted to any medical device with minimal efforts.

New concepts for mitral valve imaging.

The high complexity of the mitral valve (MV) anatomy and function is not yet fully understood. Studying especially the dynamic movement and interaction of MV components to describe MV physiology during the cardiac cycle remains a challenge. Imaging is the key to assessing details of MV disease and to studying the lesion and dysfunction of MV according to Carpentier. With the advances of computational geometrical and biomechanical MV models, improved quantification and characterization of the MV has been realized. Geometrical models can be divided into rigid and dynamic models. Both models are based on reconstruction techniques of echocardiographic or computed tomographic data sets. They allow detailed analysis of MV morphology and dynamics throughout the cardiac cycle. Biomechanical models aim to simulate the biomechanics of MV to allow for examination and analysis of the MV structure with blood flow. Two categories of biomechanical MV models can be distinguished: structural models and fluid-structure interaction (FSI) models. The complex structure and dynamics of MV apparatus throughout the cardiac cycle can be analyzed with different types of computational models. These represent substantial progress in the diagnosis of structural heart disease since MV morphology and dynamics can be studied in unprecedented detail. It is conceivable that MV modeling will contribute significantly to the understanding of the MV.

Automated quantitative 3-dimensional modeling of the aortic valve and root by 3-dimensional transesophageal echocardiography in normals, aortic regurgitation, and aortic stenosis: comparison to computed tomography in normals and clinical implications.

BACKGROUND: We tested the ability of a novel automated 3-dimensional (3D) algorithm to model and quantify the aortic root from 3D transesophageal echocardiography (TEE) and computed tomographic (CT) data. METHODS AND RESULTS: We compared the quantitative parameters obtained by automated modeling from 3D TEE (n=20) and CT data (n=20) to those made by 2D TEE and targeted 2D from 3D TEE and CT in patients without valve disease (normals). We also compared the automated 3D TEE measurements in severe aortic stenosis (n=14), dilated root without aortic regurgitation (n=15), and dilated root with aortic regurgitation (n=20). The automated 3D TEE sagittal annular diameter was significantly greater than the 2D TEE measurements (P=0.004). This was also true for the 3D TEE and CT coronal annular diameters (P<0.01). The average 3D TEE and CT annular diameter was greater than both their respective 2D and 3D sagittal diameters (P<0.001). There was no significant difference in 2D and 3D measurements of the sinotubular junction and sinus of valsalva diameters (P>0.05) in normals, but these were significantly different (P<0.05) in abnormals. The 3 automated intercommissural distance and leaflet length and height did not show significant differences in the normals (P>0.05), but all 3 were significantly different compared with the abnormal group (P<0.05). The automated 3D annulus commissure coronary ostia distances in normals showed significant difference between 3D TEE and CT (P<0.05); also, these parameters by automated 3D TEE were significantly different in abnormal (P<0.05). Finally, the automated 3D measurements showed excellent reproducibility for all parameters. CONCLUSIONS: Automated quantitative 3D modeling of the aortic root from 3D TEE or CT data is technically feasible and provides unique data that may aid surgical and transcatheter interventions.

Non-invasive hemodynamic assessment of aortic coarctation: validation with in vivo measurements.

We propose a CFD-based approach for the non-invasive hemodynamic assessment of pre- and post-operative coarctation of aorta (CoA) patients. Under our approach, the pressure gradient across the coarctation is determined from computational modeling based on physiological principles, medical imaging data, and routine non-invasive clinical measurements. The main constituents of our approach are a reduced-order model for computing blood flow in patient-specific aortic geometries, a parameter estimation procedure for determining patient-specific boundary conditions and vessel wall parameters from non-invasive measurements, and a comprehensive pressure-drop formulation coupled with the overall reduced-order model. The proposed CFD-based algorithm is fully automatic, requiring no iterative tuning procedures for matching the computed results to observed patient data, and requires approximately 6-8 min of computation time on a standard personal computer (Intel Core2 Duo CPU, 3.06 GHz), thus making it feasible for use in a clinical setting. The initial validation studies for the pressure-drop computations have been performed on four patient datasets with native or recurrent coarctation, by comparing the results with the invasively measured peak pressure gradients recorded during routine cardiac catheterization procedure. The preliminary results are promising, with a mean absolute error of less than 2 mmHg in all the patients.

Image-based computational models for TAVI planning: from CT images to implant deployment.

Transcatheter aortic valve implantation (TAVI) is becoming the standard choice of care for non-operable patients suffering from severe aortic valve stenosis. As there is no direct view or access to the affected anatomy, accurate preoperative planning is crucial for a successful outcome. The most important decision during planning is selecting the proper implant type and size. Due to the wide variety in device sizes and types and non-circular annulus shapes, there is often no obvious choice for the specific patient. Most clinicians base their final decision on their previous experience. As a first step towards a more predictive planning, we propose an integrated method to estimate the aortic apparatus from CT images and compute implant deployment. Aortic anatomy, which includes aortic root, leaflets and calcifications, is automatically extracted using robust modeling and machine learning algorithms. Then, the finite element method is employed to calculate the deployment of a TAVI implant inside the patient-specific aortic anatomy. The anatomical model was evaluated on 198 CT images, yielding an accuracy of 1.30 +/- 0.23 mm. In eleven subjects, pre- and post-TAVI CT images were available. Errors in predicted implant deployment were of 1.74 +/- 0.40 mm in average and 1.32 mm in the aortic valve annulus region, which is almost three times lower than the average gap of 3 mm between consecutive implant sizes. Our framework may thus constitute a surrogate tool for TAVI planning.

Robust model-based 3d/3D fusion using sparse matching for minimally invasive surgery.

Classical surgery is being disrupted by minimally invasive and transcatheter procedures. As there is no direct view or access to the affected anatomy, advanced imaging techniques such as 3D C-arm CT and C-arm fluoroscopy are routinely used for intra-operative guidance. However, intra-operative modalities have limited image quality of the soft tissue and a reliable assessment of the cardiac anatomy can only be made by injecting contrast agent, which is harmful to the patient and requires complex acquisition protocols. We propose a novel sparse matching approach for fusing high quality pre-operative CT and non-contrasted, non-gated intra-operative C-arm CT by utilizing robust machine learning and numerical optimization techniques. Thus, high-quality patient-specific models can be extracted from the pre-operative CT and mapped to the intra-operative imaging environment to guide minimally invasive procedures. Extensive quantitative experiments demonstrate that our model-based fusion approach has an average execution time of 2.9 s, while the accuracy lies within expert user confidence intervals.

Learning without labeling: domain adaptation for ultrasound transducer localization.

The fusion of image data from trans-esophageal echography (TEE) and X-ray fluoroscopy is attracting increasing interest in minimally-invasive treatment of structural heart disease. In order to calculate the needed transform between both imaging systems, we employ a discriminative learning based approach to localize the TEE transducer in X-ray images. Instead of time-consuming manual labeling, we generate the required training data automatically from a single volumetric image of the transducer. In order to adapt this system to real X-ray data, we use unlabeled fluoroscopy images to estimate differences in feature space density and correct covariate shift by instance weighting. An evaluation on more than 1900 images reveals that our approach reduces detection failures by 95% compared to cross validation on the test set and improves the localization error from 1.5 to 0.8 mm. Due to the automatic generation of training data, the proposed system is highly flexible and can be adapted to any medical device with minimal efforts.

An integrated framework for finite-element modeling of mitral valve biomechanics from medical images: application to MitralClip intervention planning.

Treatment of mitral valve (MV) diseases requires comprehensive clinical evaluation and therapy personalization to optimize outcomes. Finite-element models (FEMs) of MV physiology have been proposed to study the biomechanical impact of MV repair, but their translation into the clinics remains challenging. As a step towards this goal, we present an integrated framework for finite-element modeling of the MV closure based on patient-specific anatomies and boundary conditions. Starting from temporal medical images, we estimate a comprehensive model of the MV apparatus dynamics, including papillary tips, using a machine-learning approach. A detailed model of the open MV at end-diastole is then computed, which is finally closed according to a FEM of MV biomechanics. The motion of the mitral annulus and papillary tips are constrained from the image data for increased accuracy. A sensitivity analysis of our system shows that chordae rest length and boundary conditions have a significant influence upon the simulation results. We quantitatively test the generalization of our framework on 25 consecutive patients. Comparisons between the simulated closed valve and ground truth show encouraging results (average point-to-mesh distance: 1.49 ± 0.62 mm) but also the need for personalization of tissue properties, as illustrated in three patients. Finally, the predictive power of our model is tested on one patient who underwent MitralClip by comparing the simulated intervention with the real outcome in terms of MV closure, yielding promising prediction. By providing an integrated way to perform MV simulation, our framework may constitute a surrogate tool for model validation and therapy planning.

Complete valvular heart apparatus model from 4D cardiac CT.

The cardiac valvular apparatus, composed of the aortic, mitral, pulmonary and tricuspid valves, is an essential part of the anatomical, functional and hemodynamic characteristics of the heart and the cardiovascular system as a whole. Valvular heart diseases often involve multiple dysfunctions and require joint assessment and therapy of the valves. In this paper, we propose a complete and modular patient-specific model of the cardiac valvular apparatus estimated from 4D cardiac CT data. A new constrained Multi-linear Shape Model (cMSM), conditioned by anatomical measurements, is introduced to represent the complex spatio-temporal variation of the heart valves. The cMSM is exploited within a learning-based framework to efficiently estimate the patient-specific valve parameters from cine images. Experiments on 64 4D cardiac CT studies demonstrate the performance and clinical potential of the proposed method. Our method enables automatic quantitative evaluation of the complete valvular apparatus based on non-invasive imaging techniques. In conjunction with existent patient-specific chamber models, the presented valvular model enables personalized computation modeling and realistic simulation of the entire cardiac system.

Hemodynamic assessment of pre- and post-operative aortic coarctation from MRI.

Coarctation of the aorta (CoA), is a congenital defect characterized by a severe narrowing of the aorta, usually distal to the aortic arch. The treatment options include surgical repair, stent implantation, and balloon angioplasty. In order to evaluate the physiological significance of the pre-operative coarctation and to assess the post-operative results, the hemodynamic analysis is usually performed by measuring the pressure gradient (deltaP) across the coarctation site via invasive cardiac catheterization. The measure of success is reduction of the (deltaP > 20 mmHg) systolic blood pressure gradient. In this paper, we propose a non-invasive method based on Computational Fluid Dynamics and MR imaging to estimate the pre- and post-operative hemodynamics for both native and recurrent coarctation patients. High correlation of our results and catheter measurements is shown on corresponding pre- and post-operative examination of 5 CoA patients.

Ultrasound and fluoroscopic images fusion by autonomous ultrasound probe detection.

New minimal-invasive interventions such as transcatheter valve procedures exploit multiple imaging modalities to guide tools (fluoroscopy) and visualize soft tissue (transesophageal echocardiography (TEE)). Currently, these complementary modalities are visualized in separate coordinate systems and on separate monitors creating a challenging clinical workflow. This paper proposes a novel framework for fusing TEE and fluoroscopy by detecting the pose of the TEE probe in the fluoroscopic image. Probe pose detection is challenging in fluoroscopy and conventional computer vision techniques are not well suited. Current research requires manual initialization or the addition of fiducials. The main contribution of this paper is autonomous six DoF pose detection by combining discriminative learning techniques with a fast binary template library. The pose estimation problem is reformulated to incrementally detect pose parameters by exploiting natural invariances in the image. The theoretical contribution of this paper is validated on synthetic, phantom and in vivo data. The practical application of this technique is supported by accurate results (< 5 mm in-plane error) and computation time of 0.5s.

Model-based fusion of multi-modal volumetric images: application to transcatheter valve procedures.

Minimal invasive procedures such as transcatheter valve interventions are substituting conventional surgical techniques. Thus, novel operating rooms have been designed to augment traditional surgical equipment with advanced imaging systems to guide the procedures. We propose a novel method to fuse pre-operative and intra-operative information by jointly estimating anatomical models from multiple image modalities. Thereby high-quality patient-specific models are integrated into the imaging environment of operating rooms to guide cardiac interventions. Robust and fast machine learning techniques are utilized to guide the estimation process. Our method integrates both the redundant and complementary multimodal information to achieve a comprehensive modeling and simultaneously reduce the estimation uncertainty. Experiments performed on 28 patients with pairs of multimodal volumetric data are used to demonstrate high quality intra-operative patient-specific modeling of the aortic valve with a precision of 1.09mm in TEE and 1.73mm in 3D C-arm CT. Within a processing time of 10 seconds we additionally obtain model sensitive mapping between the pre- and intraoperative images.

Robust physically-constrained modeling of the mitral valve and subvalvular apparatus.

Mitral valve (MV) is often involved in cardiac diseases, with various pathological patterns that require a systemic view of the entire MV apparatus. Due to its complex shape and dynamics, patient-specific modeling of the MV constitutes a particular challenge. We propose a novel approach for personalized modeling of the dynamic MV and its subvalvular apparatus that ensures temporal consistency over the cardiac sequence and provides realistic deformations. The idea is to detect the anatomical MV components under constraints derived from the biomechanical properties of the leaflets. This is achieved by a robust two-step alternate algorithm that combines discriminative learning and leaflet biomechanics. Extensive evaluation on 200 transesophageal echochardiographic sequences showed an average Hausdorff error of 5.1 mm at a speed of 9 sec, which constitutes an improvement of up to 11.5% compared to purely data driven approaches. Clinical evaluation on 42 subjects showed, that the proposed fully-automatic approach could provide discriminant biomarkers to detect and quantify remodeling of annulus and leaflets in functional mitral regurgitation.

Patient-specific modelling of whole heart anatomy, dynamics and haemodynamics from four-dimensional cardiac CT images.

There is a growing need for patient-specific and holistic modelling of the heart to support comprehensive disease assessment and intervention planning as well as prediction of therapeutic outcomes. We propose a patient-specific model of the whole human heart, which integrates morphology, dynamics and haemodynamic parameters at the organ level. The modelled cardiac structures are robustly estimated from four-dimensional cardiac computed tomography (CT), including all four chambers and valves as well as the ascending aorta and pulmonary artery. The patient-specific geometry serves as an input to a three-dimensional Navier-Stokes solver that derives realistic haemodynamics, constrained by the local anatomy, along the entire heart cycle. We evaluated our framework with various heart pathologies and the results correlate with relevant literature reports.