Multi-modal medical Transformers: A meta-analysis for medical image segmentation in oncology.

Multi-modal medical image segmentation is a crucial task in oncology that enables the precise localization and quantification of tumors. The aim of this work is to present a meta-analysis of the use of multi-modal medical Transformers for medical image segmentation in oncology, specifically focusing on multi-parametric MR brain tumor segmentation (BraTS2021), and head and neck tumor segmentation using PET-CT images (HECKTOR2021). The multi-modal medical Transformer architectures presented in this work exploit the idea of modality interaction schemes based on visio-linguistic representations: (i) single-stream, where modalities are jointly processed by one Transformer encoder, and (ii) multiple-stream, where the inputs are encoded separately before being jointly modeled. A total of fourteen multi-modal architectures are evaluated using different ranking strategies based on dice similarity coefficient (DSC) and average symmetric surface distance (ASSD) metrics. In addition, cost indicators such as the number of trainable parameters and the number of multiply-accumulate operations (MACs) are reported. The results demonstrate that multi-path hybrid CNN-Transformer-based models improve segmentation accuracy when compared to traditional methods, but come at the cost of increased computation time and potentially larger model size.

Development of Radiomic-Based Model to Predict Clinical Outcomes in Non-Small Cell Lung Cancer Patients Treated with Immunotherapy.

Purpose: We aimed to assess the ability of radiomics features extracted from baseline (PET/CT0) and follow-up PET/CT scans, as well as their evolution (delta-radiomics), to predict clinical outcome (durable clinical benefit (DCB), progression, response to therapy, OS and PFS) in non-small cell lung cancer (NSCLC) patients treated with immunotherapy. Methods: 83 NSCLC patients treated with immunotherapy who underwent a baseline PET/CT were retrospectively included. Response was assessed at 6−8 weeks (PET/CT1) using PERCIST criteria and at 3 months with iPERCIST (PET/CT2) or RECIST 1.1 criteria using CT. The predictive performance of clinical parameters (CP), standard PET metrics (SUV, Metabolic Tumor volume, Total Lesion Glycolysis), delta-radiomics and PET and CT radiomics features extracted at baseline and during follow-up were studied. Seven multivariate models with different combinations of CP and radiomics were trained on a subset of patients (75%) using least absolute shrinkage, selection operator (LASSO) and random forest classification with 10-fold cross-validation to predict outcome. Model validation was performed on the remaining patients (25%). Overall and progression-free survival was also performed by Kaplan−Meier survival analysis. Results: Numerous radiomics and delta-radiomics parameters had a high individual predictive value of patient outcome with areas under receiver operating characteristics curves (AUCs) >0.80. Their performance was superior to that of CP and standard PET metrics. Several multivariate models were also promising, especially for the prediction of progression (AUCs of 1 and 0.96 for the training and testing subsets with the PET-CT model (PET/CT0)) or DCB (AUCs of 0.85 and 0.83 with the PET-CT-CP model (PET/CT0)). Conclusions: Delta-radiomics and radiomics features extracted from baseline and follow-up PET/CT images could predict outcome in NSCLC patients treated with immunotherapy and identify patients who would benefit from this new standard. These data reinforce the rationale for the use of advanced image analysis of PET/CT scans to further improve personalized treatment management in advanced NSCLC.

FFCD 1709-SIRTCI phase II trial: Selective internal radiation therapy plus Xelox, Bevacizumab and Atezolizumab in liver-dominant metastatic colorectal cancer.

Immune checkpoint inhibitors (ICI) have high efficacy in metastatic colorectal cancer (mCRC) with microsatellite instability (MSI) but not in microsatellite stable (MSS) tumour due to the low tumour mutational burden. Selective internal radiation therapy (SIRT) could enhance neoantigen production thus triggering systemic anti-tumoral immune response (abscopal effect). In addition, Oxalipatin can induce immunogenic cell death and Bevacizumab can decrease the exhaustion of tumour infiltrating lymphocyte. In combination, these treatments could act synergistically to sensitize MSS mCRCs to ICI SIRTCI is a prospective, multicentre, open-label, phase II, non-comparative single-arm study evaluating the efficacy and safety of SIRT plus Xelox, Bevacizumab and Atezolizumab (anti-programmed death-ligand 1) in patients with liver-dominant MSS mCRC. The primary objective is progression-free survival at 9 months. The main inclusion criteria are patients with MSS mCRC with liver-dominant disease, initially unresectable disease and with no prior oncologic treatment for metastatic disease. The trial started in November 2020 and has included 10 out of the 52 planned patients.

Accurate Tumor Delineation vs. Rough Volume of Interest Analysis for 18F-FDG PET/CT Radiomics-Based Prognostic Modeling inNon-Small Cell Lung Cancer.

BACKGROUND: The aim of this work was to investigate the ability of building prognostic models in non-small cell lung cancer (NSCLC) using radiomic features from positron emission tomography and computed tomography with 2-deoxy-2-[fluorine-18]fluoro-d-glucose (18F-FDG PET/CT) images based on a "rough" volume of interest (VOI) containing the tumor instead of its accurate delineation, which is a significant time-consuming bottleneck of radiomics analyses. METHODS: A cohort of 138 patients with stage II-III NSCLC treated with radiochemotherapy recruited retrospectively (n = 87) and prospectively (n = 51) was used. Two approaches were compared: firstly, the radiomic features were extracted from the delineated primary tumor volumes in both PET (using the automated fuzzy locally adaptive Bayesian, FLAB) and CT (using a semi-automated approach with 3D Slicer™) components. Both delineations were carried out within previously manually defined "rough" VOIs containing the tumor and the surrounding tissues, which were exploited for the second approach: the same features were extracted from this alternative VOI. Both sets for features were then combined with the clinical variables and processed through the same machine learning (ML) pipelines using the retrospectively recruited patients as the training set and the prospectively recruited patients as the testing set. Logistic regression (LR), random forest (RF), and support vector machine (SVM), as well as their consensus through averaging the output probabilities, were considered for feature selection and modeling for overall survival (OS) prediction as a binary classification (either median OS or 6 months OS). The resulting models were compared in terms of balanced accuracy, sensitivity, and specificity. RESULTS: Overall, better performance was achieved using the features from delineated tumor volumes. This was observed consistently across ML algorithms and for the two clinical endpoints. However, the loss of performance was not significant, especially when a consensus of the three ML algorithms was considered (0.89 vs. 0.88 and 0.78 vs. 0.77). CONCLUSION: Our findings suggest that it is feasible to achieve similar levels of prognostic accuracy in radiomics-based modeling by relying on a faster and easier VOI definition, skipping a time-consuming tumor delineation step, thus facilitating automation of the whole radiomics workflow. The associated cost is a loss of performance in the resulting models, although this loss can be greatly mitigated when a consensus of several models is relied upon.

Comparison and Fusion of Machine Learning Algorithms for Prospective Validation of PET/CT Radiomic Features Prognostic Value in Stage II-III Non-Small Cell Lung Cancer.

Machine learning (ML) algorithms for selecting and combining radiomic features into multiparametric prediction models have become popular; however, it has been shown that large variations in performance can be obtained by relying on different approaches. The purpose of this study was to evaluate the potential benefit of combining different algorithms into an improved consensus for the final prediction, as it has been shown in other fields. Methods: The evaluation was carried out in the context of the use of radiomics from 18F-FDG PET/CT images for predicting outcome in stage II-III Non-Small Cell Lung Cancer. A cohort of 138 patients was exploited for the present analysis. Eighty-seven patients had been previously recruited retrospectively for another study and were used here for training and internal validation. We also used data from prospectively recruited patients (n = 51) for testing. Three different machine learning pipelines relying on embedded feature selection were trained to predict overall survival (OS) as a binary classification: Support Vector machines (SVMs), Random Forests (RFs), and Logistic Regression (LR). Two different clinical endpoints were investigated: median OS or OS shorter than 6 months. The fusion of the three approaches was implemented using two different strategies: majority voting on the binary outputs or averaging of the output probabilities. Results: Our results confirm previous findings, highlighting that different ML pipelines select different sets of features and reach different classification performances (accuracy in the testing set ranging between 63% and 67% for median OS, and between 75% and 80% for OS < 6 months). Generating a consensus improved the performance for both endpoints; with the probabilities averaging strategy outperforming the majority voting (accuracy of 78% vs. 71% for median OS and 89 vs. 84% for OS < 6 months). Overall, the performance of these radiomic-based models outperformed the standard clinical staging in both endpoints (accuracy of 58% and 53% accuracy in the testing set for each endpoint). Conclusion: Although obtained in a small cohort of patients, our results suggest that a consensus of machine learning algorithms can improve performance in the context of radiomics. The resulting prognostic stratification in the prospective testing cohort is higher than when relying on the clinical stage. This could be of interest for clinical practice as it could help to identify patients with higher risk amongst stage II and III patients, who could benefit from intensified treatment and/or more frequent follow-up after treatment.

Radiogenomics in Colorectal Cancer.

The steady improvement of high-throughput technologies greatly facilitates the implementation of personalized precision medicine. Characterization of tumor heterogeneity through image-derived features-radiomics and genetic profile modifications-genomics, is a rapidly evolving field known as radiogenomics. Various radiogenomics studies have been dedicated to colorectal cancer so far, highlighting the potential of these approaches to enhance clinical decision-making. In this review, a general outline of colorectal radiogenomics literature is provided, discussing the current limitations and suggested further developments.

Radiomics in PET/CT: Current Status and Future AI-Based Evolutions.

This short review aims at providing the readers with an update on the current status, as well as future perspectives in the quickly evolving field of radiomics applied to the field of PET/CT imaging. Numerous pitfalls have been identified in study design, data acquisition, segmentation, features calculation and modeling by the radiomics community, and these are often the same issues across all image modalities and clinical applications, however some of these are specific to PET/CT (and SPECT/CT) imaging and therefore the present paper focuses on those. In most cases, recommendations and potential methodological solutions do exist and should therefore be followed to improve the overall quality and reproducibility of published studies. In terms of future evolutions, the techniques from the larger field of artificial intelligence (AI), including those relying on deep neural networks (also known as deep learning) have already shown impressive potential to provide solutions, especially in terms of automation, but also to maybe fully replace the tools the radiomics community has been using until now in order to build the usual radiomics workflow. Some important challenges remain to be addressed before the full impact of AI may be realized but overall the field has made striking advances over the last few years and it is expected advances will continue at a rapid pace.

Can alternative PET reconstruction schemes improve the prognostic value of radiomic features in non-small cell lung cancer?

PURPOSE: To evaluate the potential benefit of using alternative reconstruction schemes of PET images for the prognostic value of radiomic features. METHODS: Patients (n=91) with non-small cell lung cancer were prospectively included. All had a PET/CT examination before treatment. Three different PET images were reconstructed for each patient: the standard clinical protocol (i.e., 4×4×4 mm3 voxels, 5mm Gaussian filter, denoted '200G5'), as well as using smaller voxels (i.e., 2×2×2 mm3 with a larger reconstruction matrix, denoted 400G1) and/or 1mm post-reconstruction Gaussian filter, denoted 200G1). Metabolic volumes of the primary tumors were semi-automatically delineated on the PET images and IBSI compliant radiomic features (intensity, shape, textural) were extracted. First, the distributions of 200G1 and 400G1 features were compared to the reference clinical protocol (200G5) through Bland-Altman tests and the use of linear mixed models. Then, the prognostic value of the features from each of the 3 reconstructions was evaluated in a univariate analysis, through their stratification power in Kaplan-Meier curves through a threshold set at the median. RESULTS: The 3 reconstructions led to different distributions for most of the features. The larger shifts and standard deviations of differences was observed between 200G5 and 400G1, which was also confirmed through linear mixed models. However, these relatively important differences in distributions did not translate into a significant impact on the stratification power of the features in terms of prognosis, although a trend in decreasing prognostic value could be observed (smaller number of features with HR above 2, overall lower HR values). Most prognostic features displayed high correlation with either volume or SUVmax, although there was great variability of prognostic value for similar levels of correlation with these basic metrics. CONCLUSIONS: Using smaller voxels or less strong filtering options in the reconstruction settings of PET images compared to the standard clinical protocols led to different distributions of the resulting radiomic features. However, the hierarchy between patients according to these distributions remained overall the same and therefore the resulting stratification power of the radiomic features was not significantly altered. These results should be compared to those obtained in the context of other pathologies where radiomic features displaying lower correlation with volume or SUVmax may have predictive value, such as in cervical cancer.

Transcriptomics in cancer revealed by Positron Emission Tomography radiomics.

Metabolic images from Positron Emission Tomography (PET) are used routinely for diagnosis, follow-up or treatment planning purposes of cancer patients. In this study we aimed at determining if radiomic features extracted from 18F-Fluoro Deoxy Glucose (FDG) PET images could mirror tumor transcriptomics. In this study we analyzed 45 patients with locally advanced head and neck cancer (H&N) that underwent FDG-PET scans at the time of diagnosis and transcriptome analysis using RNAs from both cancer and healthy tissues on microarrays. Association between PET radiomics and transcriptomics was carried out with the Genomica software and a functional annotation was used to associate PET radiomics, gene expression and altered biological pathways. We identified relationships between PET radiomics and genes involved in cell-cycle, disease, DNA repair, extracellular matrix organization, immune system, metabolism or signal transduction pathways, according to the Reactome classification. Our results suggest that these FDG PET radiomic features could be used to infer tissue gene expression and cellular pathway activity in H&N cancers. These observations strengthen the value of radiomics as a promising approach to personalize treatments through targeting tumor-specific molecular processes.

Qualitative and quantitative reproducibility of [18]fluoromethycholine PET/computed tomography in prostate cancer.

BACKGROUND: Recurrence occurs in more than 50% of prostate cancer. To be effective, treatments require precise localization of tumor cells. [F]fluoromethylcholine ([18F]FCH) PET/computed tomography (CT) is currently used to restage disease in cases of biochemical relapse. To be used for therapy response as has been suggested, repeatability limits of PET derived indices need to be established. OBJECTIVE: The aim of our study was to prospectively assess the qualitative and quantitative reproducibility [18F]FCH PET/CT in prostate cancer. METHODS: Patients with histologically proven prostate cancer referred for initial staging or restaging were prospectively included. All patients underwent two [18F]FCH PET/CTs in the same conditions within a maximum of 3 weeks' time. We studied the repeatability of the visual report and the repeatability of SUVmax and its evolution over the acquisition time in lesions, liver and vascular background. Statistical analysis was performed using the Bland-Altman approach. RESULTS: Twenty-one patients were included. Reporting repeatability was excellent with 97.8% of concordance. Mean repeatability of SUVmax considering all times and all lesions was 2.2% ± 20. Evolution of SUVmax was unpredictable, either increasing or decreasing over the acquisition time, both for lesions and for physiological activity. CONCLUSION: Our study demonstrated that visual report of [18F]FCH PET/CT was very reproducible and that the repeatability limits of SUVmax was similar to those of other PET radiotracers. An SUVmax difference of more than 40% should be considered as representing a treatment response effect. Change of SUVmax during the acquisition time varied and should not be considered as an interpretation criterion.

Are radiomics ready for clinical prime-time in PET/CT imaging?

Over the last few years the field of radiomics has been gaining ground in the field of nuclear medicine and in multimodality imaging. Within this context, numerous studies have exploited the potential interest of radiomics in clinical practice, within the diagnostic field as well as in prognostic and predictive modeling of patient response. Although these studies have showed some interesting results, there are also persistent conflicting conclusions. Most of these studies suffer from consistently low number of patients, lack of external dataset-based validation of most frequently single center determined models and non-standardized calculation of radiomics features. In the future, clear identification of the most pertinent applications of clinical utilization in combination with multi-center studies will allow more concrete clinical applications of radiomics to be identified. In addition, the recent interest of artificial intelligence which can completely change the current paradigm of radiomics use in clinical practice needs to be integrated within this framework. This paper highlights the main areas of radiomics applications considered up to date and provides an insight on the main issues and potential solutions in order to allow a potential future integration in clinical practice.

Integrating radiomics into clinical trial design.

In radiomics, quantitative features that describe phenotypic tumor characteristics are derived from radiographic images. Because radiomics generates information from routine medical images, it is a powerful way to non-invasively examine the spatial and temporal heterogeneity of disease, and thus has potential to significantly impact clinical trial design, execution, and ultimately patient care. The aim of this review article is to discuss how radiomics may address some of the current challenges in clinical randomized control trials, and the difficulties of integrating robust and repeatable radiomics analysis into trial design. Each step of the radiomics process, including image acquisition and reconstruction, image segmentation, feature extraction, and computational analysis, requires extensive standardization in order to be successfully incorporated into clinical trials and inform clinical decision making. By addressing these challenges, the potential of radiomics may be realized.

Artificial intelligence, machine (deep) learning and radio(geno)mics: definitions and nuclear medicine imaging applications.

Techniques from the field of artificial intelligence, and more specifically machine (deep) learning methods, have been core components of most recent developments in the field of medical imaging. They are already being exploited or are being considered to tackle most tasks, including image reconstruction, processing (denoising, segmentation), analysis and predictive modelling. In this review we introduce and define these key concepts and discuss how the techniques from this field can be applied to nuclear medicine imaging applications with a particular focus on radio(geno)mics.

Reoxygenation during radiotherapy in intermediate-risk prostate cancer.

Hypoxia is a major risk factor of prostate cancer radioresistance. We evaluated hypoxia non-invasively, using 18F-Misonidazole PET/CT prior to radiotherapy and after a dose of 20 Gy in intermediate-risk prostate cancer patients. Decreased hypoxic volumes were observed in all patients, suggesting that radiotherapy induces early prostate tumor reoxygenation.

Use of baseline 18F-FDG PET scan to identify initial sub-volumes with local failure after concomitant radio-chemotherapy in head and neck cancer.

INTRODUCTION: Head and neck squamous cell carcinoma (HNSCC) treated by radio-chemotherapy have a significant local recurrence rate. It has been previously suggested that 18F-FDG PET could identify the high uptake areas that can be potential targets for dose boosting. The purpose of this study was to compare the location of initial hypermetabolic regions on baseline scans with the metabolic relapse sites after radio-chemotherapy in HNSCC. RESULTS: The initial functional tumor volume was significantly higher for patients with proven local recurrence or residual disease (23.5 cc vs. 8.9 cc; p = 0.0005). The overlap between baseline and follow-up sub-volumes were moderate with an overlap fraction ranging from 0.52 to 0.39 between R40 and I30 to I60. CONCLUSION: In our study the overlap between baseline and post-therapeutic metabolic tumor sub-volumes was only moderate. These results need to be investigated in a larger cohort acquired with a more standardized patient repositioning protocol for sequential PET imaging. METHODS: Pre and post treatment PET/CT scans of ninety four HNSCC patients treated with radio-chemotherapy were retrospectively reviewed. Follow-up 18F-FDG PET/CT images were registered to baseline scans using a rigid body transformation. Seven metabolic tumor sub-volumes were obtained on the baseline scans using a fixed percentage of SUVmax (I30, I40, I50, I60, I70, I80, and I90) and were subsequently compared with two post-treatment sub-volumes (R40, R90) in 38 cases of local recurrence or residual metabolic disease. Overlap fraction, Dice and Jaccard indices, common volume/baseline volume and common volume/recurrent volume were used to determine the overlap of the different estimated sub-volumes.

Evaluation of tumor hypoxia prior to radiotherapy in intermediate-risk prostate cancer using 18F-fluoromisonidazole PET/CT: a pilot study.

PURPOSE: Hypoxia is a major factor in prostate cancer aggressiveness and radioresistance. Predicting which patients might be bad candidates for radiotherapy may help better personalize treatment decisions in intermediate-risk prostate cancer patients. We assessed spatial distribution of 18F-Misonidazole (FMISO) PET/CT uptake in the prostate prior to radiotherapy treatment. MATERIALS AND METHODS: Intermediate-risk prostate cancer patients about to receive high-dose (>74 Gy) radiotherapy to the prostate without hormonal treatment were prospectively recruited between 9/2012 and 10/2014. Prior to radiotherapy, all patients underwent a FMISO PET/CT as well as a MRI and 18F-choline-PET. 18F-choline and FMISO-positive volumes were semi-automatically determined using the fuzzy locally adaptive Bayesian (FLAB) method. In FMISO-positive patients, a dynamic analysis of early tumor uptake was performed. Group differences were assessed using the Wilcoxon signed rank test. Parameters were correlated using Spearman rank correlation. RESULTS: Of 27 patients (median age 76) recruited to the study, 7 and 9 patients were considered positive at 2.5h and 3.5h FMISO PET/CT respectively. Median SUVmax and SUVmax tumor to muscle (T/M) ratio were respectively 3.4 and 3.6 at 2.5h, and 3.2 and 4.4 at 3.5h. The median FMISO-positive volume was 1.1 ml. CONCLUSIONS: This is the first study regarding hypoxia imaging using FMISO in prostate cancer showing that a small FMISO-positive volume was detected in one third of intermediate-risk prostate cancer patients.

Reliability of PET/CT Shape and Heterogeneity Features in Functional and Morphologic Components of Non-Small Cell Lung Cancer Tumors: A Repeatability Analysis in a Prospective Multicenter Cohort.

The main purpose of this study was to assess the reliability of shape and heterogeneity features in both the PET and the low-dose CT components of PET/CT. A secondary objective was to investigate the impact of image quantization. Methods: A Health Insurance Portability and Accountability Act-compliant secondary analysis of deidentified prospectively acquired PET/CT test-retest datasets of 74 patients from multicenter Merck and American College of Radiology Imaging Network trials was performed. Metabolically active volumes were automatically delineated on PET with a fuzzy locally adaptive bayesian algorithm. Software was used to semiautomatically delineate the anatomic volumes on the low-dose CT component. Two quantization methods were considered: a quantization into a set number of bins (quantization B) and an alternative quantization with bins of fixed width (quantization W). Four shape descriptors, 10 first-order metrics, and 26 textural features were evaluated. Bland-Altman analysis was used to quantify repeatability. Features were subsequently categorized as very reliable, reliable, moderately reliable, or poorly reliable with respect to the corresponding volume variability. Results: Repeatability was highly variable among features. Numerous metrics were identified as poorly or moderately reliable. Others were reliable or very reliable in both modalities and in all categories (shape and first-, second-, and third-order metrics). Image quantization played a major role in feature repeatability. Features were more reliable in PET with quantization B, whereas quantization W showed better results in CT. Conclusion: The test-retest repeatability of shape and heterogeneity features in PET and low-dose CT varied greatly among metrics. The level of repeatability also depended strongly on the quantization step, with different optimal choices for each modality. The repeatability of PET and low-dose CT features should be carefully considered when selecting metrics to build multiparametric models.

Comparison of Tumor Uptake Heterogeneity Characterization Between Static and Parametric 18F-FDG PET Images in Non-Small Cell Lung Cancer.

UNLABELLED: (18)F-FDG PET is well established in the field of oncology for diagnosis and staging purposes and is increasingly being used to assess therapeutic response and prognosis. Many quantitative indices can be used to characterize tumors on (18)F-FDG PET images, such as SUVmax, metabolically active tumor volume (MATV), total lesion glycolysis, and, more recently, the proposed intratumor uptake heterogeneity features. Although most PET data considered within this context concern the analysis of activity distribution using images obtained from a single static acquisition, parametric images generated from dynamic acquisitions and reflecting radiotracer kinetics may provide additional information. The purpose of this study was to quantify differences between volumetry, uptake, and heterogeneity features extracted from static and parametric PET images of non-small cell lung carcinoma (NSCLC) in order to provide insight on the potential added value of parametric images. METHODS: Dynamic (18)F-FDG PET/CT was performed on 20 therapy-naive NSCLC patients for whom primary surgical resection was planned. Both static and parametric PET images were analyzed, with quantitative parameters (MATV, SUVmax, SUVmean, heterogeneity) being extracted from the segmented tumors. Differences were investigated using Spearman rank correlation and Bland-Altman analysis. RESULTS: MATV was slightly smaller on static images (-2% ± 7%), but the difference was not significant (P = 0.14). All derived parameters, including those characterizing tumor functional heterogeneity, correlated strongly between static and parametric images (r = 0.70-0.98, P ≤ 0.0006), exhibiting differences of less than ±25%. CONCLUSION: In NSCLC primary tumors, parametric and static baseline (18)F-FDG PET images provided strongly correlated quantitative features for both standard (MATV, SUVmax, SUVmean) and heterogeneity quantification. Consequently, heterogeneity quantification on parametric images does not seem to provide significant complementary information compared with static SUV images.