Semiautomated segmentation of hepatocellular carcinoma tumors with MRI using convolutional neural networks.

OBJECTIVE: To assess the performance of convolutional neural networks (CNNs) for semiautomated segmentation of hepatocellular carcinoma (HCC) tumors on MRI. METHODS: This retrospective single-center study included 292 patients (237 M/55F, mean age 61 years) with pathologically confirmed HCC between 08/2015 and 06/2019 and who underwent MRI before surgery. The dataset was randomly divided into training (n = 195), validation (n = 66), and test sets (n = 31). Volumes of interest (VOIs) were manually placed on index lesions by 3 independent radiologists on different sequences (T2-weighted imaging [WI], T1WI pre-and post-contrast on arterial [AP], portal venous [PVP], delayed [DP, 3 min post-contrast] and hepatobiliary phases [HBP, when using gadoxetate], and diffusion-weighted imaging [DWI]). Manual segmentation was used as ground truth to train and validate a CNN-based pipeline. For semiautomated segmentation of tumors, we selected a random pixel inside the VOI, and the CNN provided two outputs: single slice and volumetric outputs. Segmentation performance and inter-observer agreement were analyzed using the 3D Dice similarity coefficient (DSC). RESULTS: A total of 261 HCCs were segmented on the training/validation sets, and 31 on the test set. The median lesion size was 3.0 cm (IQR 2.0-5.2 cm). Mean DSC (test set) varied depending on the MRI sequence with a range between 0.442 (ADC) and 0.778 (high b-value DWI) for single-slice segmentation; and between 0.305 (ADC) and 0.667 (T1WI pre) for volumetric-segmentation. Comparison between the two models showed better performance in single-slice segmentation, with statistical significance on T2WI, T1WI-PVP, DWI, and ADC. Inter-observer reproducibility of segmentation analysis showed a mean DSC of 0.71 in lesions between 1 and 2 cm, 0.85 in lesions between 2 and 5 cm, and 0.82 in lesions > 5 cm. CONCLUSION: CNN models have fair to good performance for semiautomated HCC segmentation, depending on the sequence and tumor size, with better performance for the single-slice approach. Refinement of volumetric approaches is needed in future studies. KEY POINTS: • Semiautomated single-slice and volumetric segmentation using convolutional neural networks (CNNs) models provided fair to good performance for hepatocellular carcinoma segmentation on MRI. • CNN models' performance for HCC segmentation accuracy depends on the MRI sequence and tumor size, with the best results on diffusion-weighted imaging and T1-weighted imaging pre-contrast, and for larger lesions.

Response heterogeneity as a new biomarker of treatment response in patients with neuroendocrine tumors.

Aim: The RAISE project aimed to find a surrogate end point to predict treatment response early in patients with enteropancreatic neuroendocrine tumors (NET). Response heterogeneity, defined as the coexistence of responding and non-responding lesions, has been proposed as a predictive marker for progression-free survival (PFS) in patients with NETs. Patients & methods: Computerized tomography scans were analyzed from patients with multiple lesions in CLARINET (NCT00353496; n = 148/204). Cox regression analyses evaluated association between response heterogeneity, estimated using the standard deviation of the longest diameter ratio of target lesions, and NET progression. Results: Greater response heterogeneity at a given visit was associated with earlier progression thereafter: week 12 hazard ratio (HR; 95% confidence interval): 1.48 (1.20-1.82); p < 0.001; n = 148; week 36: 1.72 (1.32-2.24); p < 0.001; n = 108. HRs controlled for sum of longest diameter ratio: week 12: 1.28 (1.04-1.59); p = 0.020 and week 36: 1.81 (1.20-2.72); p = 0.005. Conclusion: Response heterogeneity independently predicts PFS in patients with enteropancreatic NETs. Further validation is required.

Neuroendocrine tumors (NET) are rare, slow-growing cancers that can grow in various parts of the body. By understanding how NETs are responding to treatment, doctors can choose the best treatment for a patient and monitor whether the treatment needs to be changed. Treatment response is determined using 'Response Evaluation Criteria in Solid Tumors (RECIST)': a technique which measures the size of tumors to assess whether they are shrinking. However, RECIST is not always useful in NETs, which grow slowly and rarely shrink. 'Response heterogeneity' describes the situation in which some tumors respond well to treatment, while other tumors in the same patient do not. Response heterogeneity may be important in understanding how tumors are responding to treatment and predicting outcomes for patients. Until now, the link between response heterogeneity and treatment response has not been studied in patients with NETs. The RAISE project examined data from a clinical trial of patients with NETs treated with lanreotide. In RAISE, response heterogeneity was estimated using imaging scans of NETs. Response heterogeneity was compared with factors such as tumor size and amounts of certain molecules found in the blood, to see how well response heterogeneity could predict outcomes for patients with NETs. In this study, response heterogeneity was linked with worse outcomes for patients. Therefore, it may be useful in understanding how NETs respond to treatment. Further research is needed in a different group of patients with NETs, and in patients receiving other treatments, to better understand response heterogeneity.

The use of deep learning models to predict progression-free survival in patients with neuroendocrine tumors.

Aim: The RAISE project assessed whether deep learning could improve early progression-free survival (PFS) prediction in patients with neuroendocrine tumors. Patients & methods: Deep learning models extracted features from CT scans from patients in CLARINET (NCT00353496) (n = 138/204). A Cox model assessed PFS prediction when combining deep learning with the sum of longest diameter ratio (SLDr) and logarithmically transformed CgA concentration (logCgA), versus SLDr and logCgA alone. Results: Deep learning models extracted features other than lesion shape to predict PFS at week 72. No increase in performance was achieved with deep learning versus SLDr and logCgA models alone. Conclusion: Deep learning models extracted relevant features to predict PFS, but did not improve early prediction based on SLDr and logCgA.

Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG-PET/CT for total metabolic tumour volume prediction using a convolutional neural network.

PURPOSE: Lymphoma lesion detection and segmentation on whole-body FDG-PET/CT are a challenging task because of the diversity of involved nodes, organs or physiological uptakes. We sought to investigate the performances of a three-dimensional (3D) convolutional neural network (CNN) to automatically segment total metabolic tumour volume (TMTV) in large datasets of patients with diffuse large B cell lymphoma (DLBCL). METHODS: The dataset contained pre-therapy FDG-PET/CT from 733 DLBCL patients of 2 prospective LYmphoma Study Association (LYSA) trials. The first cohort (n = 639) was used for training using a 5-fold cross validation scheme. The second cohort (n = 94) was used for external validation of TMTV predictions. Ground truth masks were manually obtained after a 41% SUVmax adaptive thresholding of lymphoma lesions. A 3D U-net architecture with 2 input channels for PET and CT was trained on patches randomly sampled within PET/CTs with a summed cross entropy and Dice similarity coefficient (DSC) loss. Segmentation performance was assessed by the DSC and Jaccard coefficients. Finally, TMTV predictions were validated on the second independent cohort. RESULTS: Mean DSC and Jaccard coefficients (± standard deviation) in the validations set were 0.73 ± 0.20 and 0.68 ± 0.21, respectively. An underestimation of mean TMTV by - 12 mL (2.8%) ± 263 was found in the validation sets of the first cohort (P = 0.27). In the second cohort, an underestimation of mean TMTV by - 116 mL (20.8%) ± 425 was statistically significant (P = 0.01). CONCLUSION: Our CNN is a promising tool for automatic detection and segmentation of lymphoma lesions, despite slight underestimation of TMTV. The fully automatic and open-source features of this CNN will allow to increase both dissemination in routine practice and reproducibility of TMTV assessment in lymphoma patients.

Prognostic value of anthropometric measures extracted from whole-body CT using deep learning in patients with non-small-cell lung cancer.

INTRODUCTION: The aim of the study was to extract anthropometric measures from CT by deep learning and to evaluate their prognostic value in patients with non-small-cell lung cancer (NSCLC). METHODS: A convolutional neural network was trained to perform automatic segmentation of subcutaneous adipose tissue (SAT), visceral adipose tissue (VAT), and muscular body mass (MBM) from low-dose CT images in 189 patients with NSCLC who underwent pretherapy PET/CT. After a fivefold cross-validation in a subset of 35 patients, anthropometric measures extracted by deep learning were normalized to the body surface area (BSA) to control the various patient morphologies. VAT/SAT ratio and clinical parameters were included in a Cox proportional-hazards model for progression-free survival (PFS) and overall survival (OS). RESULTS: Inference time for a whole volume was about 3 s. Mean Dice similarity coefficients in the validation set were 0.95, 0.93, and 0.91 for SAT, VAT, and MBM, respectively. For PFS prediction, T-stage, N-stage, chemotherapy, radiation therapy, and VAT/SAT ratio were associated with disease progression on univariate analysis. On multivariate analysis, only N-stage (HR = 1.7 [1.2-2.4]; p = 0.006), radiation therapy (HR = 2.4 [1.0-5.4]; p = 0.04), and VAT/SAT ratio (HR = 10.0 [2.7-37.9]; p < 0.001) remained significant prognosticators. For OS, male gender, smoking status, N-stage, a lower SAT/BSA ratio, and a higher VAT/SAT ratio were associated with mortality on univariate analysis. On multivariate analysis, male gender (HR = 2.8 [1.2-6.7]; p = 0.02), N-stage (HR = 2.1 [1.5-2.9]; p < 0.001), and the VAT/SAT ratio (HR = 7.9 [1.7-37.1]; p < 0.001) remained significant prognosticators. CONCLUSION: The BSA-normalized VAT/SAT ratio is an independent predictor of both PFS and OS in NSCLC patients. KEY POINTS: • Deep learning will make CT-derived anthropometric measures clinically usable as they are currently too time-consuming to calculate in routine practice. • Whole-body CT-derived anthropometrics in non-small-cell lung cancer are associated with progression-free survival and overall survival. • A priori medical knowledge can be implemented in the neural network loss function calculation.

The STOIC2021 COVID-19 AI challenge: Applying reusable training methodologies to private data.

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.

Integrating deep learning CT-scan model, biological and clinical variables to predict severity of COVID-19 patients.

The SARS-COV-2 pandemic has put pressure on intensive care units, so that identifying predictors of disease severity is a priority. We collect 58 clinical and biological variables, and chest CT scan data, from 1003 coronavirus-infected patients from two French hospitals. We train a deep learning model based on CT scans to predict severity. We then construct the multimodal AI-severity score that includes 5 clinical and biological variables (age, sex, oxygenation, urea, platelet) in addition to the deep learning model. We show that neural network analysis of CT-scans brings unique prognosis information, although it is correlated with other markers of severity (oxygenation, LDH, and CRP) explaining the measurable but limited 0.03 increase of AUC obtained when adding CT-scan information to clinical variables. Here, we show that when comparing AI-severity with 11 existing severity scores, we find significantly improved prognosis performance; AI-severity can therefore rapidly become a reference scoring approach.

Comparative performances of machine learning methods for classifying Crohn Disease patients using genome-wide genotyping data.

Crohn Disease (CD) is a complex genetic disorder for which more than 140 genes have been identified using genome wide association studies (GWAS). However, the genetic architecture of the trait remains largely unknown. The recent development of machine learning (ML) approaches incited us to apply them to classify healthy and diseased people according to their genomic information. The Immunochip dataset containing 18,227 CD patients and 34,050 healthy controls enrolled and genotyped by the international Inflammatory Bowel Disease genetic consortium (IIBDGC) has been re-analyzed using a set of ML methods: penalized logistic regression (LR), gradient boosted trees (GBT) and artificial neural networks (NN). The main score used to compare the methods was the Area Under the ROC Curve (AUC) statistics. The impact of quality control (QC), imputing and coding methods on LR results showed that QC methods and imputation of missing genotypes may artificially increase the scores. At the opposite, neither the patient/control ratio nor marker preselection or coding strategies significantly affected the results. LR methods, including Lasso, Ridge and ElasticNet provided similar results with a maximum AUC of 0.80. GBT methods like XGBoost, LightGBM and CatBoost, together with dense NN with one or more hidden layers, provided similar AUC values, suggesting limited epistatic effects in the genetic architecture of the trait. ML methods detected near all the genetic variants previously identified by GWAS among the best predictors plus additional predictors with lower effects. The robustness and complementarity of the different methods are also studied. Compared to LR, non-linear models such as GBT or NN may provide robust complementary approaches to identify and classify genetic markers.