A framework for validating AI in precision medicine: considerations from the European ITFoC consortium.

BACKGROUND: Artificial intelligence (AI) has the potential to transform our healthcare systems significantly. New AI technologies based on machine learning approaches should play a key role in clinical decision-making in the future. However, their implementation in health care settings remains limited, mostly due to a lack of robust validation procedures. There is a need to develop reliable assessment frameworks for the clinical validation of AI. We present here an approach for assessing AI for predicting treatment response in triple-negative breast cancer (TNBC), using real-world data and molecular -omics data from clinical data warehouses and biobanks. METHODS: The European "ITFoC (Information Technology for the Future Of Cancer)" consortium designed a framework for the clinical validation of AI technologies for predicting treatment response in oncology. RESULTS: This framework is based on seven key steps specifying: (1) the intended use of AI, (2) the target population, (3) the timing of AI evaluation, (4) the datasets used for evaluation, (5) the procedures used for ensuring data safety (including data quality, privacy and security), (6) the metrics used for measuring performance, and (7) the procedures used to ensure that the AI is explainable. This framework forms the basis of a validation platform that we are building for the "ITFoC Challenge". This community-wide competition will make it possible to assess and compare AI algorithms for predicting the response to TNBC treatments with external real-world datasets. CONCLUSIONS: The predictive performance and safety of AI technologies must be assessed in a robust, unbiased and transparent manner before their implementation in healthcare settings. We believe that the consideration of the ITFoC consortium will contribute to the safe transfer and implementation of AI in clinical settings, in the context of precision oncology and personalized care.

Co-registered optical coherence tomography and X-ray angiography for the prediction of fractional flow reserve.

Cardiovascular disease (CVD) stands as the leading global cause of mortality, and coronary artery disease (CAD) has the highest prevalence, contributing to 42% of these fatalities. Recognizing the constraints inherent in the anatomical assessment of CAD, Fractional Flow Reserve (FFR) has emerged as a pivotal functional diagnostic metric. Herein, we assess the potential of employing an ensemble approach with deep neural networks (DNN) to predict invasively measured Fractional Flow Reserve (FFR) using raw anatomical data extracted from both optical coherence tomography (OCT) and X-ray coronary angiography (XA). In this study, we used a challenging dataset, with 46% of the lesions falling within the FFR range of 0.75 to 0.85. Despite this complexity, our model achieved an accuracy of 84.3%, demonstrating a sensitivity of 87.5% and a specificity of 81.4%. Our results demonstrate that incorporating both OCT and XA signals, co-registered, as inputs for the DNN model leads to an important increase in overall accuracy.

Deep neural networks for ECG-free cardiac phase and end-diastolic frame detection on coronary angiographies.

Invasive coronary angiography (ICA) is the gold standard in Coronary Artery Disease (CAD) imaging. Detection of the end-diastolic frame (EDF) and, in general, cardiac phase detection on each temporal frame of a coronary angiography acquisition is of significant importance for the anatomical and non-invasive functional assessment of CAD. This task is generally performed via manual frame selection or semi-automated selection based on simultaneously acquired ECG signals - thus introducing the requirement of simultaneous ECG recordings. In this paper, we evaluate the performance of a purely image based workflow relying on deep neural networks for fully automated cardiac phase and EDF detection on coronary angiographies. A first deep neural network (DNN), trained to detect coronary arteries, is employed to preselect a subset of frames in which coronary arteries are well visible. A second DNN predicts cardiac phase labels for each frame. Only in the training and evaluation phases for the second DNN, ECG signals are used to provide ground truth labels for each angiographic frame. The networks were trained on 56,655 coronary angiographies from 6820 patients and evaluated on 20,780 coronary angiographies from 6261 patients. No exclusion criteria related to patient state (stable or acute CAD), previous interventions (PCI or CABG), or pathology were formulated. Cardiac phase detection had an accuracy of 98.8 %, a sensitivity of 99.3 % and a specificity of 97.6 % on the evaluation set. EDF prediction had a precision of 98.4 % and a recall of 97.9 %. Several sub-group analyses were performed, indicating that the cardiac phase detection performance is largely independent from acquisition angles, the heart rate of the patient, and the angiographic view (LCA / RCA). The average execution time of cardiac phase detection for one angiographic series was on average less than five seconds on a standard workstation. We conclude that the proposed image based workflow potentially obviates the need for manual frame selection and ECG acquisition, representing a relevant step towards automated CAD assessment.