Le Cœur en Sabot: shape associations with adverse events in repaired tetralogy of Fallot.

BACKGROUND: Maladaptive remodelling mechanisms occur in patients with repaired tetralogy of Fallot (rToF) resulting in a cycle of metabolic and structural changes. Biventricular shape analysis may indicate mechanisms associated with adverse events independent of pulmonary regurgitant volume index (PRVI). We aimed to determine novel remodelling patterns associated with adverse events in patients with rToF using shape and function analysis. METHODS: Biventricular shape and function were studied in 192 patients with rToF (median time from TOF repair to baseline evaluation 13.5 years). Linear discriminant analysis (LDA) and principal component analysis (PCA) were used to identify shape differences between patients with and without adverse events. Adverse events included death, arrhythmias, and cardiac arrest with median follow-up of 10 years. RESULTS: LDA and PCA showed that shape characteristics pertaining to adverse events included a more circular left ventricle (LV) (decreased eccentricity), dilated (increased sphericity) LV base, increased right ventricular (RV) apical sphericity, and decreased RV basal sphericity. Multivariate LDA showed that the optimal discriminative model included only RV apical ejection fraction and one PCA mode associated with a more circular and dilated LV base (AUC = 0.77). PRVI did not add value, and shape changes associated with increased PRVI were not predictive of adverse outcomes. CONCLUSION: Pathological remodelling patterns in patients with rToF are significantly associated with adverse events, independent of PRVI. Mechanisms related to incident events include LV basal dilation with a reduced RV apical ejection fraction.

Machine Learning for Surgical Phase Recognition: A Systematic Review.

OBJECTIVE: To provide an overview of ML models and data streams utilized for automated surgical phase recognition. BACKGROUND: Phase recognition identifies different steps and phases of an operation. ML is an evolving technology that allows analysis and interpretation of huge data sets. Automation of phase recognition based on data inputs is essential for optimization of workflow, surgical training, intraoperative assistance, patient safety, and efficiency. METHODS: A systematic review was performed according to the Cochrane recommendations and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. PubMed, Web of Science, IEEExplore, GoogleScholar, and CiteSeerX were searched. Literature describing phase recognition based on ML models and the capture of intraoperative signals during general surgery procedures was included. RESULTS: A total of 2254 titles/abstracts were screened, and 35 full-texts were included. Most commonly used ML models were Hidden Markov Models and Artificial Neural Networks with a trend towards higher complexity over time. Most frequently used data types were feature learning from surgical videos and manual annotation of instrument use. Laparoscopic cholecystectomy was used most commonly, often achieving accuracy rates over 90%, though there was no consistent standardization of defined phases. CONCLUSIONS: ML for surgical phase recognition can be performed with high accuracy, depending on the model, data type, and complexity of surgery. Different intraoperative data inputs such as video and instrument type can successfully be used. Most ML models still require significant amounts of manual expert annotations for training. The ML models may drive surgical workflow towards standardization, efficiency, and objectiveness to improve patient outcome in the future. REGISTRATION PROSPERO: CRD42018108907.

Surgical phase and instrument recognition: how to identify appropriate dataset splits.

PURPOSE: Machine learning approaches can only be reliably evaluated if training, validation, and test data splits are representative and not affected by the absence of classes. Surgical workflow and instrument recognition are two tasks that are complicated in this manner, because of heavy data imbalances resulting from different length of phases and their potential erratic occurrences. Furthermore, sub-properties like instrument (co-)occurrence are usually not particularly considered when defining the split. METHODS: We present a publicly available data visualization tool that enables interactive exploration of dataset partitions for surgical phase and instrument recognition. The application focuses on the visualization of the occurrence of phases, phase transitions, instruments, and instrument combinations across sets. Particularly, it facilitates assessment of dataset splits, especially regarding identification of sub-optimal dataset splits. RESULTS: We performed analysis of the datasets Cholec80, CATARACTS, CaDIS, M2CAI-workflow, and M2CAI-tool using the proposed application. We were able to uncover phase transitions, individual instruments, and combinations of surgical instruments that were not represented in one of the sets. Addressing these issues, we identify possible improvements in the splits using our tool. A user study with ten participants demonstrated that the participants were able to successfully solve a selection of data exploration tasks. CONCLUSION: In highly unbalanced class distributions, special care should be taken with respect to the selection of an appropriate dataset split because it can greatly influence the assessments of machine learning approaches. Our interactive tool allows for determination of better splits to improve current practices in the field. The live application is available at https://cardio-ai.github.io/endovis-ml/ .

Surgical training of minimally invasive mitral valve repair on a patient-specific simulator improves surgical skills.

OBJECTIVES: Minimally invasive mitral valve repair (MVR) is considered one of the most challenging operations in cardiac surgery and requires much practice and experience. Simulation-based surgical training might be a method to support the learning process and help to flatten the steep learning curve of novices. The purpose of this study was to show the possible effects on learning of surgical training using a high-fidelity simulator with patient-specific mitral valve replicas. METHODS: Twenty-five participants were recruited to perform MVR on anatomically realistic valve models during different training sessions. After every session their performance was evaluated by a surgical expert regarding accuracy and duration for each step. A second blinded rater similarly assessed the performance after the study. Through repeated documentation of those parameters, their progress in learning was analysed, and gains in proficiency were evaluated. RESULTS: Participants showed significant performance enhancements in terms of both accuracy and time. Their surgical skills showed sizeable improvements after only 1 session. For example, the time to implant neo-chordae decreased by 24.64% (354 s-264 s, P < 0.001) and the time for annuloplasty by 4.01% (54 s-50 s, P = 0.165), whereas the number of irregular stitches for annuloplasty decreased from 52% to 24%.The significance of simulation-based surgical training as a tool for acquiring and training surgical skills was reviewed positively. CONCLUSIONS: The results of this study indicate that simulation-based surgical training is a valuable and effective method for learning reconstructive techniques of minimally invasive MVR and overall general dexterity.The novel learning and training options should be implemented in the surgical traineeship for systematic teaching of various surgical skills.

Perfect Match: Radiomics and Artificial Intelligence in Cardiac Imaging.

Cardiovascular diseases remain a significant health burden, with imaging modalities like echocardiography, cardiac computed tomography, and cardiac magnetic resonance imaging playing a crucial role in diagnosis and prognosis. However, the inherent heterogeneity of these diseases poses challenges, necessitating advanced analytical methods like radiomics and artificial intelligence. Radiomics extracts quantitative features from medical images, capturing intricate patterns and subtle variations that may elude visual inspection. Artificial intelligence techniques, including deep learning, can analyze these features to generate knowledge, define novel imaging biomarkers, and support diagnostic decision-making and outcome prediction. Radiomics and artificial intelligence thus hold promise for significantly enhancing diagnostic and prognostic capabilities in cardiac imaging, paving the way for more personalized and effective patient care. This review explores the synergies between radiomics and artificial intelligence in cardiac imaging, following the radiomics workflow and introducing concepts from both domains. Potential clinical applications, challenges, and limitations are discussed, along with solutions to overcome them.

Combined conventional factors and the radiomics signature of coronary plaque texture could improve cardiac risk prediction.

OBJECTIVES: This study aims to investigate how radiomics analysis can help understand the association between plaque texture, epicardial adipose tissue (EAT), and cardiovascular risk. Working with a Photon-counting CT, which exhibits enhanced feature stability, offers the potential to advance radiomics analysis and enable its integration into clinical routines. METHODS: Coronary plaques were manually segmented in this retrospective, single-centre study and radiomic features were extracted using pyradiomics. The study population was divided into groups according to the presence of high-risk plaques (HRP), plaques with at least 50% stenosis, plaques with at least 70% stenosis, or triple-vessel disease. A combined group with patients exhibiting at least one of these risk factors was formed. Random forest feature selection identified differentiating features for the groups. EAT thickness and density were measured and compared with feature selection results. RESULTS: A total number of 306 plaques from 61 patients (mean age 61 years +/- 8.85 [standard deviation], 13 female) were analysed. Plaques of patients with HRP features or relevant stenosis demonstrated a higher presence of texture heterogeneity through various radiomics features compared to patients with only an intermediate stenosis degree. While EAT thickness did not significantly differ, affected patients showed significantly higher mean densities in the 50%, HRP, and combined groups, and insignificantly higher densities in the 70% and triple-vessel groups. CONCLUSION: The combination of a higher EAT density and a more heterogeneous plaque texture might offer an additional tool in identifying patients with an elevated risk of cardiovascular events. CLINICAL RELEVANCE STATEMENT: Cardiovascular disease is the leading cause of mortality globally. Plaque composition and changes in the EAT are connected to cardiac risk. A better understanding of the interrelation of these risk indicators can lead to improved cardiac risk prediction. KEY POINTS: Cardiac plaque composition and changes in the EAT are connected to cardiac risk. Higher EAT density and more heterogeneous plaque texture are related to traditional risk indicators. Radiomics texture analysis conducted on PCCT scans can help identify patients with elevated cardiac risk.

Artificial Intelligence to derive aligned strain in cine CMR to detect patients with myocardial fibrosis: an open and scrutinizable approach.

Cine Cardiac Magnetic Resonance (CMR) is the gold standard for cardiac function evaluation, incorporating ejection fraction (EF) and strain as vital indicators of abnormal deformation. Rare pathologies like Duchenne muscular dystrophies (DMD) are monitored with repeated late gadolinium-enhanced (LGE) CMR for identification of myocardial fibrosis. However, it is judicious to reduce repeated gadolinium exposure and rather employ strain analysis from cine CMR. This solution is limited so far since full strain curves are not comparable between individual cardiac cycles and current practice mainly neglects diastolic deformation patterns. Our novel Deep Learning-based approach derives strain values aligned by key frames throughout the cardiac cycle. In a reproducibility scenario (57+82 patients), our results reveal five times more significant differences (22 vs. 4) between patients with scar and without, enhancing scar detection by +30%, improving detection of patients with preserved EF by +61%, with an overall sensitivity/specificity of 82/81%.

An ex-vivo and in-vitro dynamic simulator for surgical and transcatheter mitral valve interventions.

PURPOSE: Minimally invasive mitral valve surgery (MIMVS) and transcatheter edge-to-edge repair (TEER) are complex procedures used to treat mitral valve (MV) pathologies, but with limited training opportunities available. To enable training, a realistic hemodynamic environment is needed. In this work we aimed to develop and validate a simulator that enables investigation of MV pathologies and their repair by MIMVS and TEER in a hemodynamic setting. METHODS: Different MVs were installed in the simulator, and pressure, flow, and transesophageal echocardiographic measurements were obtained. To confirm the simulator's physiological range, we first installed a biological prosthetic, a mechanical prosthetic, and a competent excised porcine MV. Subsequently, we inserted two porcine MVs-one with induced chordae tendineae rupture and the other with a dilated annulus, along with a patient-specific silicone valve extracted from echocardiography with bi-leaflet prolapse. Finally, TEER and MIMVS procedures were conducted by experts to repair the MVs. RESULTS: Systolic pressures, cardiac outputs, and regurgitations volumes (RVol) with competent MVs were 119 ± 1 mmHg, 4.78 ± 0.16 l min-1, and 5 ± 3 ml respectively, and thus within the physiological range. In contrast, the pathological MVs displayed increased RVols. MIMVS and TEER resulted in a decrease in RVols and mitigated the severity of mitral regurgitation. CONCLUSION: Ex-vivo modelling of MV pathologies and repair procedures using the described simulator realistically replicated physiological in-vivo conditions. Furthermore, we showed the feasibility of performing MIMVS and TEER at the simulator, also at patient-specific level, thus providing new clinical perspectives in terms of training modalities and personalized planning.

Simulation of thoracic endovascular aortic repair in a perfused patient-specific model of type B aortic dissection.

PURPOSE: Complicated type B Aortic dissection is a severe aortic pathology that requires treatment through thoracic endovascular aortic repair (TEVAR). During TEVAR a stentgraft is deployed in the aortic lumen in order to restore blood flow. Due to the complicated pathology including an entry, a resulting dissection wall with potentially several re-entries, replicating this structure artificially has proven to be challenging thus far. METHODS: We developed a 3d printed, patient-specific and perfused aortic dissection phantom with a flexible dissection flap and all major branching vessels. The model was segmented from CTA images and fabricated out of a flexible material to mimic aortic wall tissue. It was placed in a pulsatile hemodynamic flow loop. Hemodynamics were investigated through pressure and flow measurements and doppler ultrasound imaging. Surgeons performed a TEVAR intervention including stentgraft deployment under fluoroscopic guidance. RESULTS: The flexible aortic dissection phantom was successfully incorporated in the hemodynamic flow loop, a systolic pressure of 112 mmHg and physiological flow of 4.05 L per minute was reached. Flow velocities were higher in true lumen with a up to 35.7 cm/s compared to the false lumen with a maximum of 13.3 cm/s, chaotic flow patterns were observed on main entry and reentry sights. A TEVAR procedure was successfully performed under fluoroscopy. The position of the stentgraft was confirmed using CTA imaging. CONCLUSIONS: This perfused in-vitro phantom allows for detailed investigation of the complex inner hemodynamics of aortic dissections on a patient-specific level and enables the simulation of TEVAR procedures in a real endovascular operating environment. Therefore, it could provide a dynamic platform for future surgical training and research.

Prediction of diagnosis and diastolic filling pressure by AI-enhanced cardiac MRI: a modelling study of hospital data.

BACKGROUND: With increasing numbers of patients and novel drugs for distinct causes of systolic and diastolic heart failure, automated assessment of cardiac function is important. We aimed to provide a non-invasive method to predict diagnosis of patients undergoing cardiac MRI (cMRI) and to obtain left ventricular end-diastolic pressure (LVEDP). METHODS: For this modelling study, patients who had undergone cardiac catheterisation at University Hospital Heidelberg (Heidelberg, Germany) between July 15, 2004 and March 16, 2023, were identified, as were individual left ventricular pressure measurements. We used existing patient data from routine cardiac diagnostics. From this initial group, we extracted patients who had been diagnosed with ischaemic cardiomyopathy, dilated cardiomyopathy, hypertrophic cardiomyopathy, or amyloidosis, as well as control individuals with no structural phenotype. Data were pseudonymised and only processed within the university hospital's AI infrastructure. We used the data to build different models to predict either demographic (ie, AI-age and AI-sex), diagnostic (ie, AI-coronary artery disease and AI-cardiomyopathy [AI-CMP]), or functional parameters (ie, AI-LVEDP). We randomly divided our datasets via computer into training, validation, and test datasets. AI-CMP was not compared with other models, but was validated in a prospective setting. Benchmarking was also done. FINDINGS: 66 936 patients who had undergone cardiac catheterisation at University Hospital Heidelberg were identified, with more than 183 772 individual left ventricular pressure measurements. We extracted 4390 patients from this initial group, of whom 1131 (25·8%) had been diagnosed with ischaemic cardiomyopathy, 1064 (24·2%) had been diagnosed with dilated cardiomyopathy, 816 (18·6%) had been diagnosed with hypertrophic cardiomyopathy, 202 (4·6%) had been diagnosed with amyloidosis, and 1177 (26·7%) were control individuals with no structural phenotype. The core cohort only included patients with cardiac catherisation and cMRI within 30 days, and emergency cases were excluded. AI-sex was able to predict patient sex with areas under the receiver operating characteristic curves (AUCs) of 0·78 (95% CI 0·77-0·78) and AI-age was able to predict patient age with a mean absolute error of 7·86 years (7·77-7·95), with a Pearson correlation of 0·57 (95% CI 0·56-0·57). The AUCs for the classification tasks ranged between 0·82 (95% CI 0·79-0·84) for ischaemic cardiomyopathy and 0·92 (0·91-0·94) for hypertrophic cardiomyopathy. INTERPRETATION: Our AI models could be easily integrated into clinical practice and provide added value to the information content of cMRI, allowing for disease classification and prediction of diastolic function. FUNDING: Informatics for Life initiative of the Klaus-Tschira Foundation, German Center for Cardiovascular Research, eCardiology section of the German Cardiac Society, and AI Health Innovation Cluster Heidelberg.

Interactive visual exploration of surgical process data.

PURPOSE: Integrated operating rooms provide rich sources of temporal information about surgical procedures, which has led to the emergence of surgical data science. However, little emphasis has been put on interactive visualization of such temporal datasets to gain further insights. Our goal is to put heterogeneous data sequences in relation to better understand the workflows of individual procedures as well as selected subsets, e.g., with respect to different surgical phase distributions and surgical instrument usage patterns. METHODS: We developed a reusable web-based application design to analyze data derived from surgical procedure recordings. It consists of aggregated, synchronized visualizations for the original temporal data as well as for derived information, and includes tailored interaction techniques for selection and filtering. To enable reproducibility, we evaluated it across four types of surgeries from two openly available datasets (HeiCo and Cholec80). User evaluation has been conducted with twelve students and practitioners with surgical and technical background. RESULTS: The evaluation showed that the application has the complexity of an expert tool (System Usability Score of 57.73) but allowed the participants to solve various analysis tasks correctly (78.8% on average) and to come up with novel hypotheses regarding the data. CONCLUSION: The novel application supports postoperative expert-driven analysis, improving the understanding of surgical workflows and the underlying datasets. It facilitates analysis across multiple synchronized views representing information from different data sources and, thereby, advances the field of surgical data science.

Surgical Rehearsal for Mitral Valve Repair: Personalizing Surgical Simulation by 3D Printing.

PURPOSE: The goal of this study was to show possible effects of performing the actual procedure of mitral valve repair (MVR) on personalized silicone models 1 day before operation. DESCRIPTION: Based on preoperative 3-dimensional echocardiography recordings, flexible 3-dimensional replicas of the depicted pathologic mitral valves could be produced and used for a simulation of reconstructive techniques analogous to the upcoming MVR procedure. We integrated this step of personalized surgical planning into the clinical routine of 6 MVR cases with 3 different surgeons. This pilot study was assessed by evaluating questionnaires and by comparing isolated surgical steps with conventional MVRs. EVALUATION: This approach was considered a better preparation for MVRs with overall positive responses from the surgeons. Simulation helped reduce the time of initial inspection of the valve because of better understanding of the valve's pathomorphologic features. Annuloplasty benefited from preoperative sizing by reducing the number of sizing attempts. CONCLUSIONS: These initial findings suggest that simulation-based surgical planning can be implemented into patients' and physicians' clinical workflow as a major technologic advancement for future MVR preparation.

Comparative evaluation of three commercially available markerless depth sensors for close-range use in surgical simulation.

PURPOSE: Minimally invasive surgeries have restricted surgical ports, demanding a high skill level from the surgeon. Surgical simulation potentially reduces this steep learning curve and additionally provides quantitative feedback. Markerless depth sensors show great promise for quantification, but most such sensors are not designed for accurate reconstruction of complex anatomical forms in close-range. METHODS: This work compares three commercially available depth sensors, namely the Intel D405, D415, and the Stereolabs Zed-Mini in the range of 12-20 cm, for use in surgical simulation. Three environments are designed that closely mimic surgical simulation, comprising planar surfaces, rigid objects, and mitral valve models of silicone and realistic porcine tissue. The cameras are evaluated on Z-accuracy, temporal noise, fill rate, checker distance, point cloud comparisons, and visual inspection of surgical scenes, across several camera settings. RESULTS: The Intel cameras show sub-mm accuracy in most static environments. The D415 fails in reconstructing valve models, while the Zed-Mini provides lesser temporal noise and higher fill rate. The D405 could reconstruct anatomical structures like the mitral valve leaflet and a ring prosthesis, but performs poorly for reflective surfaces like surgical tools and thin structures like sutures. CONCLUSION: If a high temporal resolution is needed and lower spatial resolution is acceptable, the Zed-Mini is the best choice, whereas the Intel D405 is the most suited for close-range applications. The D405 shows potential for applications like deformable registration of surfaces, but is not yet suitable for applications like real-time tool tracking or surgical skill assessment.

Denoising diffusion probabilistic models for 3D medical image generation.

Recent advances in computer vision have shown promising results in image generation. Diffusion probabilistic models have generated realistic images from textual input, as demonstrated by DALL-E 2, Imagen, and Stable Diffusion. However, their use in medicine, where imaging data typically comprises three-dimensional volumes, has not been systematically evaluated. Synthetic images may play a crucial role in privacy-preserving artificial intelligence and can also be used to augment small datasets. We show that diffusion probabilistic models can synthesize high-quality medical data for magnetic resonance imaging (MRI) and computed tomography (CT). For quantitative evaluation, two radiologists rated the quality of the synthesized images regarding "realistic image appearance", "anatomical correctness", and "consistency between slices". Furthermore, we demonstrate that synthetic images can be used in self-supervised pre-training and improve the performance of breast segmentation models when data is scarce (Dice scores, 0.91 [without synthetic data], 0.95 [with synthetic data]).

Posterior temperature optimized Bayesian models for inverse problems in medical imaging.

We present Posterior Temperature Optimized Bayesian Inverse Models (POTOBIM), an unsupervised Bayesian approach to inverse problems in medical imaging using mean-field variational inference with a fully tempered posterior. Bayesian methods exhibit useful properties for approaching inverse tasks, such as tomographic reconstruction or image denoising. A suitable prior distribution introduces regularization, which is needed to solve the ill-posed problem and reduces overfitting the data. In practice, however, this often results in a suboptimal posterior temperature, and the full potential of the Bayesian approach is not being exploited. In POTOBIM, we optimize both the parameters of the prior distribution and the posterior temperature with respect to reconstruction accuracy using Bayesian optimization with Gaussian process regression. Our method is extensively evaluated on four different inverse tasks on a variety of modalities with images from public data sets and we demonstrate that an optimized posterior temperature outperforms both non-Bayesian and Bayesian approaches without temperature optimization. The use of an optimized prior distribution and posterior temperature leads to improved accuracy and uncertainty estimation and we show that it is sufficient to find these hyperparameters per task domain. Well-tempered posteriors yield calibrated uncertainty, which increases the reliability in the predictions. Our source code is publicly available at github.com/Cardio-AI/mfvi-dip-mia.

Mutually Improved Endoscopic Image Synthesis and Landmark Detection in Unpaired Image-to-Image Translation.

The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. This can be viewed as a novel augmented reality approach, which we coined Hyperrealism in previous work. In this use case, it is of paramount importance to display objects like needles, sutures or instruments consistent in both domains while altering the style to a more tissue-like appearance. Segmentation of these objects would allow for a direct transfer, however, contouring of these, partly tiny and thin foreground objects is cumbersome and perhaps inaccurate. Instead, we propose to use landmark detection on the points when sutures pass into the tissue. This objective is directly incorporated into a CycleGAN framework by treating the performance of pre-trained detector models as an additional optimization goal. We show that a task defined on these sparse landmark labels improves consistency of synthesis by the generator network in both domains. Comparing a baseline CycleGAN architecture to our proposed extension (DetCycleGAN), mean precision (PPV) improved by +61.32, mean sensitivity (TPR) by +37.91, and mean F1 score by +0.4743. Furthermore, it could be shown that by dataset fusion, generated intra-operative images can be leveraged as additional training data for the detection network itself.

M2aia-Interactive, fast, and memory-efficient analysis of 2D and 3D multi-modal mass spectrometry imaging data.

BACKGROUND: Mass spectrometry imaging (MSI) is a label-free analysis method for resolving bio-molecules or pharmaceuticals in the spatial domain. It offers unique perspectives for the examination of entire organs or other tissue specimens. Owing to increasing capabilities of modern MSI devices, the use of 3D and multi-modal MSI becomes feasible in routine applications-resulting in hundreds of gigabytes of data. To fully leverage such MSI acquisitions, interactive tools for 3D image reconstruction, visualization, and analysis are required, which preferably should be open-source to allow scientists to develop custom extensions. FINDINGS: We introduce M2aia (MSI applications for interactive analysis in MITK), a software tool providing interactive and memory-efficient data access and signal processing of multiple large MSI datasets stored in imzML format. M2aia extends MITK, a popular open-source tool in medical image processing. Besides the steps of a typical signal processing workflow, M2aia offers fast visual interaction, image segmentation, deformable 3D image reconstruction, and multi-modal registration. A unique feature is that fused data with individual mass axes can be visualized in a shared coordinate system. We demonstrate features of M2aia by reanalyzing an N-glycan mouse kidney dataset and 3D reconstruction and multi-modal image registration of a lipid and peptide dataset of a mouse brain, which we make publicly available. CONCLUSIONS: To our knowledge, M2aia is the first extensible open-source application that enables a fast, user-friendly, and interactive exploration of large datasets. M2aia is applicable to a wide range of MSI analysis tasks.

Point detection through multi-instance deep heatmap regression for sutures in endoscopy.

PURPOSE: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. METHOD: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. RESULTS: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean [Formula: see text] of [Formula: see text] over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean [Formula: see text] of [Formula: see text] over the baseline. CONCLUSION: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/ , and the code at https://github.com/Cardio-AI/suture-detection-pytorch/ .

Unsupervised Domain Adaptation From Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks.

Anisotropic multi-slice Cardiac Magnetic Resonance (CMR) Images are conventionally acquired in patient-specific short-axis (SAX) orientation. In specific cardiovascular diseases that affect right ventricular (RV) morphology, acquisitions in standard axial (AX) orientation are preferred by some investigators, due to potential superiority in RV volume measurement for treatment planning. Unfortunately, due to the rare occurrence of these diseases, data in this domain is scarce. Recent research in deep learning-based methods mainly focused on SAX CMR images and they had proven to be very successful. In this work, we show that there is a considerable domain shift between AX and SAX images, and therefore, direct application of existing models yield sub-optimal results on AX samples. We propose a novel unsupervised domain adaptation approach, which uses task-related probabilities in an attention mechanism. Beyond that, cycle consistency is imposed on the learned patient-individual 3D rigid transformation to improve stability when automatically re-sampling the AX images to SAX orientations. The network was trained on 122 registered 3D AX-SAX CMR volume pairs from a multi-centric patient cohort. A mean 3D Dice of 0.86 ± 0.06 for the left ventricle, 0.65 ± 0.08 for the myocardium, and 0.77 ± 0.10 for the right ventricle could be achieved. This is an improvement of 25% in Dice for RV in comparison to direct application on axial slices. To conclude, our pre-trained task module has neither seen CMR images nor labels from the target domain, but is able to segment them after the domain gap is reduced. Code: https://github.com/Cardio-AI/3d-mri-domain-adaptation.

3D Printing, Computational Modeling, and Artificial Intelligence for Structural Heart Disease.

Structural heart disease (SHD) is a new field within cardiovascular medicine. Traditional imaging modalities fall short in supporting the needs of SHD interventions, as they have been constructed around the concept of disease diagnosis. SHD interventions disrupt traditional concepts of imaging in requiring imaging to plan, simulate, and predict intraprocedural outcomes. In transcatheter SHD interventions, the absence of a gold-standard open cavity surgical field deprives physicians of the opportunity for tactile feedback and visual confirmation of cardiac anatomy. Hence, dependency on imaging in periprocedural guidance has led to evolution of a new generation of procedural skillsets, concept of a visual field, and technologies in the periprocedural planning period to accelerate preclinical device development, physician, and patient education. Adaptation of 3-dimensional (3D) printing in clinical care and procedural planning has demonstrated a reduction in early-operator learning curve for transcatheter interventions. Integration of computation modeling to 3D printing has accelerated research and development understanding of fluid mechanics within device testing. Application of 3D printing, computational modeling, and ultimately incorporation of artificial intelligence is changing the landscape of physician training and delivery of patient-centric care. Transcatheter structural heart interventions are requiring in-depth periprocedural understanding of cardiac pathophysiology and device interactions not afforded by traditional imaging metrics.