Combining a deep learning model with clinical data better predicts hepatocellular carcinoma behavior following surgery.

Hepatocellular carcinoma (HCC) is among the most common cancers worldwide, and tumor recurrence following liver resection or transplantation is one of the highest contributors to mortality in HCC patients after surgery. Using artificial intelligence (AI), we developed an interdisciplinary model to predict HCC recurrence and patient survival following surgery. We collected whole-slide H&E images, clinical variables, and follow-up data from 300 patients with HCC who underwent transplant and 169 patients who underwent resection at the Cleveland Clinic. A deep learning model was trained to predict recurrence-free survival (RFS) and disease-specific survival (DSS) from the H&E-stained slides. Repeated cross-validation splits were used to compute robust C-index estimates, and the results were compared to those obtained by fitting a Cox proportional hazard model using only clinical variables. While the deep learning model alone was predictive of recurrence and survival among patients in both cohorts, integrating the clinical and histologic models significantly increased the C-index in each cohort. In every subgroup analyzed, we found that a combined clinical and deep learning model better predicted post-surgical outcome in HCC patients compared to either approach independently.

Deep learning predicts patients outcome and mutations from digitized histology slides in gastrointestinal stromal tumor.

Risk assessment of gastrointestinal stromal tumor (GIST) according to the AFIP/Miettinen classification and mutational profiling are major tools for patient management. However, the AFIP/Miettinen classification depends heavily on mitotic counts, which is laborious and sometimes inconsistent between pathologists. It has also been shown to be imperfect in stratifying patients. Molecular testing is costly and time-consuming, therefore, not systematically performed in all countries. New methods to improve risk and molecular predictions are hence crucial to improve the tailoring of adjuvant therapy. We have built deep learning (DL) models on digitized HES-stained whole slide images (WSI) to predict patients' outcome and mutations. Models were trained with a cohort of 1233 GIST and validated on an independent cohort of 286 GIST. DL models yielded comparable results to the Miettinen classification for relapse-free-survival prediction in localized GIST without adjuvant Imatinib (C-index=0.83 in cross-validation and 0.72 for independent testing). DL splitted Miettinen intermediate risk GIST into high/low-risk groups (p value = 0.002 in the training set and p value = 0.29 in the testing set). DL models achieved an area under the receiver operating characteristic curve (AUC) of 0.81, 0.91, and 0.71 for predicting mutations in KIT, PDGFRA and wild type, respectively, in cross-validation and 0.76, 0.90, and 0.55 in independent testing. Notably, PDGFRA exon18 D842V mutation, which is resistant to Imatinib, was predicted with an AUC of 0.87 and 0.90 in cross-validation and independent testing, respectively. Additionally, novel histological criteria predictive of patients' outcome and mutations were identified by reviewing the tiles selected by the models. As a proof of concept, our study showed the possibility of implementing DL with digitized WSI and may represent a reproducible way to improve tailoring therapy and precision medicine for patients with GIST.

Pacpaint: a histology-based deep learning model uncovers the extensive intratumor molecular heterogeneity of pancreatic adenocarcinoma.

Two tumor (Classical/Basal) and stroma (Inactive/active) subtypes of Pancreatic adenocarcinoma (PDAC) with prognostic and theragnostic implications have been described. These molecular subtypes were defined by RNAseq, a costly technique sensitive to sample quality and cellularity, not used in routine practice. To allow rapid PDAC molecular subtyping and study PDAC heterogeneity, we develop PACpAInt, a multi-step deep learning model. PACpAInt is trained on a multicentric cohort (n = 202) and validated on 4 independent cohorts including biopsies (surgical cohorts n = 148; 97; 126 / biopsy cohort n = 25), all with transcriptomic data (n = 598) to predict tumor tissue, tumor cells from stroma, and their transcriptomic molecular subtypes, either at the whole slide or tile level (112 µm squares). PACpAInt correctly predicts tumor subtypes at the whole slide level on surgical and biopsies specimens and independently predicts survival. PACpAInt highlights the presence of a minor aggressive Basal contingent that negatively impacts survival in 39% of RNA-defined classical cases. Tile-level analysis ( > 6 millions) redefines PDAC microheterogeneity showing codependencies in the distribution of tumor and stroma subtypes, and demonstrates that, in addition to the Classical and Basal tumors, there are Hybrid tumors that combine the latter subtypes, and Intermediate tumors that may represent a transition state during PDAC evolution.

Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer.

Triple-negative breast cancer (TNBC) is a rare cancer, characterized by high metastatic potential and poor prognosis, and has limited treatment options. The current standard of care in nonmetastatic settings is neoadjuvant chemotherapy (NACT), but treatment efficacy varies substantially across patients. This heterogeneity is still poorly understood, partly due to the paucity of curated TNBC data. Here we investigate the use of machine learning (ML) leveraging whole-slide images and clinical information to predict, at diagnosis, the histological response to NACT for early TNBC women patients. To overcome the biases of small-scale studies while respecting data privacy, we conducted a multicentric TNBC study using federated learning, in which patient data remain secured behind hospitals' firewalls. We show that local ML models relying on whole-slide images can predict response to NACT but that collaborative training of ML models further improves performance, on par with the best current approaches in which ML models are trained using time-consuming expert annotations. Our ML model is interpretable and is sensitive to specific histological patterns. This proof of concept study, in which federated learning is applied to real-world datasets, paves the way for future biomarker discovery using unprecedentedly large datasets.

An artificial intelligence model predicts the survival of solid tumour patients from imaging and clinical data.

BACKGROUND: The need for developing new biomarkers is increasing with the emergence of many targeted therapies. Artificial Intelligence (AI) algorithms have shown great promise in the medical imaging field to build predictive models. We developed a prognostic model for solid tumour patients using AI on multimodal data. PATIENTS AND METHODS: Our retrospective study included examinations of patients with seven different cancer types performed between 2003 and 2017 in 17 different hospitals. Radiologists annotated all metastases on baseline computed tomography (CT) and ultrasound (US) images. Imaging features were extracted using AI models and used along with the patients' and treatments' metadata. A Cox regression was fitted to predict prognosis. Performance was assessed on a left-out test set with 1000 bootstraps. RESULTS: The model was built on 436 patients and tested on 196 patients (mean age 59, IQR: 51-6, 411 men out of 616 patients). On the whole, 1147 US images were annotated with lesions delineation, and 632 thorax-abdomen-pelvis CTs (total of 301,975 slices) were fully annotated with a total of 9516 lesions. The developed model reaches an average concordance index of 0.71 (0.67-0.76, 95% CI). Using the median predicted risk as a threshold value, the model is able to significantly (log-rank test P value < 0.001) isolate high-risk patients from low-risk patients (respective median OS of 11 and 31 months) with a hazard ratio of 3.5 (2.4-5.2, 95% CI). CONCLUSION: AI was able to extract prognostic features from imaging data, and along with clinical data, allows an accurate stratification of patients' prognoses.

A deep learning model to predict RNA-Seq expression of tumours from whole slide images.

Deep learning methods for digital pathology analysis are an effective way to address multiple clinical questions, from diagnosis to prediction of treatment outcomes. These methods have also been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides has yet been performed. We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone, without expert annotation. Through its interpretable design, HE2RNA provides virtual spatialization of gene expression, as validated by CD3- and CD20-staining on an independent dataset. The transcriptomic representation learned by HE2RNA can also be transferred on other datasets, even of small size, to increase prediction performance for specific molecular phenotypes. We illustrate the use of this approach in clinical diagnosis purposes such as the identification of tumors with microsatellite instability.

Predicting Survival After Hepatocellular Carcinoma Resection Using Deep Learning on Histological Slides.

BACKGROUND AND AIMS: Standardized and robust risk-stratification systems for patients with hepatocellular carcinoma (HCC) are required to improve therapeutic strategies and investigate the benefits of adjuvant systemic therapies after curative resection/ablation. APPROACH AND RESULTS: In this study, we used two deep-learning algorithms based on whole-slide digitized histological slides (whole-slide imaging; WSI) to build models for predicting survival of patients with HCC treated by surgical resection. Two independent series were investigated: a discovery set (Henri Mondor Hospital, n = 194) used to develop our algorithms and an independent validation set (The Cancer Genome Atlas [TCGA], n = 328). WSIs were first divided into small squares ("tiles"), and features were extracted with a pretrained convolutional neural network (preprocessing step). The first deep-learning-based algorithm ("SCHMOWDER") uses an attention mechanism on tumoral areas annotated by a pathologist whereas the second ("CHOWDER") does not require human expertise. In the discovery set, c-indices for survival prediction of SCHMOWDER and CHOWDER reached 0.78 and 0.75, respectively. Both models outperformed a composite score incorporating all baseline variables associated with survival. Prognostic value of the models was further validated in the TCGA data set, and, as observed in the discovery series, both models had a higher discriminatory power than a score combining all baseline variables associated with survival. Pathological review showed that the tumoral areas most predictive of poor survival were characterized by vascular spaces, the macrotrabecular architectural pattern, and a lack of immune infiltration. CONCLUSIONS: This study shows that artificial intelligence can help refine the prediction of HCC prognosis. It highlights the importance of pathologist/machine interactions for the construction of deep-learning algorithms that benefit from expert knowledge and allow a biological understanding of their output.