A Convolutional-Transformer Model for FFR and iFR Assessment From Coronary Angiography.

The quantification of stenosis severity from X-ray catheter angiography is a challenging task. Indeed, this requires to fully understand the lesion's geometry by analyzing dynamics of the contrast material, only relying on visual observation by clinicians. To support decision making for cardiac intervention, we propose a hybrid CNN-Transformer model for the assessment of angiography-based non-invasive fractional flow-reserve (FFR) and instantaneous wave-free ratio (iFR) of intermediate coronary stenosis. Our approach predicts whether a coronary artery stenosis is hemodynamically significant and provides direct FFR and iFR estimates. This is achieved through a combination of regression and classification branches that forces the model to focus on the cut-off region of FFR (around 0.8 FFR value), which is highly critical for decision-making. We also propose a spatio-temporal factorization mechanisms that redesigns the transformer's self-attention mechanism to capture both local spatial and temporal interactions between vessel geometry, blood flow dynamics, and lesion morphology. The proposed method achieves state-of-the-art performance on a dataset of 778 exams from 389 patients. Unlike existing methods, our approach employs a single angiography view and does not require knowledge of the key frame; supervision at training time is provided by a classification loss (based on a threshold of the FFR/iFR values) and a regression loss for direct estimation. Finally, the analysis of model interpretability and calibration shows that, in spite of the complexity of angiographic imaging data, our method can robustly identify the location of the stenosis and correlate prediction uncertainty to the provided output scores.

Neural Transformers for Intraductal Papillary Mucosal Neoplasms (IPMN) Classification in MRI images.

Early detection of precancerous cysts or neoplasms, i.e., Intraductal Papillary Mucosal Neoplasms (IPMN), in pancreas is a challenging and complex task, and it may lead to a more favourable outcome. Once detected, grading IPMNs accurately is also necessary, since low-risk IPMNs can be under surveillance program, while high-risk IPMNs have to be surgically resected before they turn into cancer. Current standards (Fukuoka and others) for IPMN classification show significant intra- and inter-operator variability, beside being error-prone, making a proper diagnosis unreliable. The established progress in artificial intelligence, through the deep learning paradigm, may provide a key tool for an effective support to medical decision for pancreatic cancer. In this work, we follow this trend, by proposing a novel AI-based IPMN classifier that leverages the recent success of transformer networks in generalizing across a wide variety of tasks, including vision ones. We specifically show that our transformer-based model exploits pre-training better than standard convolutional neural networks, thus supporting the sought architectural universalism of transformers in vision, including the medical image domain and it allows for a better interpretation of the obtained results.

TwinLiverNet: Predicting TACE Treatment Outcome from CT scans for Hepatocellular Carcinoma using Deep Capsule Networks.

Predicting response to treatment plays a key role to assist radiologists in hepato-cellular carcinoma (HCC) therapy planning. The most widely used treatment for unresectable HCC is the trans-arterial chemoembolization (TACE). A complete radiological response after the first TACE is a reliable predictor of treatment favourable outcome. However, visual inspection of contrast-enhanced CT scans is time-consuming, error prone and too operator-dependent. Thus, in this paper we propose TwinLiverNet: a deep neural network that is able to predict TACE treatment outcome through learning visual cue from CT scans. TwinLiverNet, specifically, integrates 3D convolutions and capsule networks and is designed to process simultaneously late arterial and delayed phases from contrast-enhanced CTs. Experimental results carried out on a dataset consisting of 126 HCC lesions show that TwinLiverNet reaches an average accuracy of 82% in predicting complete response to TACE treatment. Furthermore, combining multiple CT phases (specifically, late arterial and delayed ones) yields a performance increase of over 12 percent points. Finally, the introduction of capsule layers into the model avoids the model to overfit, while enhancing accuracy.Clinical relevance- TwinLiverNet supports radiologists in visual inspection of CT scans to assess TACE treatment outcome, while reducing inter-operator variability.

Deep learning for automated skeletal bone age assessment in X-ray images.

Skeletal bone age assessment is a common clinical practice to investigate endocrinology, genetic and growth disorders in children. It is generally performed by radiological examination of the left hand by using either the Greulich and Pyle (G&P) method or the Tanner-Whitehouse (TW) one. However, both clinical procedures show several limitations, from the examination effort of radiologists to (most importantly) significant intra- and inter-operator variability. To address these problems, several automated approaches (especially relying on the TW method) have been proposed; nevertheless, none of them has been proved able to generalize to different races, age ranges and genders. In this paper, we propose and test several deep learning approaches to assess skeletal bone age automatically; the results showed an average discrepancy between manual and automatic evaluation of about 0.8 years, which is state-of-the-art performance. Furthermore, this is the first automated skeletal bone age assessment work tested on a public dataset and for all age ranges, races and genders, for which the source code is available, thus representing an exhaustive baseline for future research in the field. Beside the specific application scenario, this paper aims at providing answers to more general questions about deep learning on medical images: from the comparison between deep-learned features and manually-crafted ones, to the usage of deep-learning methods trained on general imagery for medical problems, to how to train a CNN with few images.