Predicting gastric cancer tumor mutational burden from histopathological images using multimodal deep learning.

Tumor mutational burden (TMB) is a significant predictive biomarker for selecting patients that may benefit from immune checkpoint inhibitor therapy. Whole exome sequencing is a common method for measuring TMB; however, its clinical application is limited by the high cost and time-consuming wet-laboratory experiments and bioinformatics analysis. To address this challenge, we downloaded multimodal data of 326 gastric cancer patients from The Cancer Genome Atlas, including histopathological images, clinical data and various molecular data. Using these data, we conducted a comprehensive analysis to investigate the relationship between TMB, clinical factors, gene expression and image features extracted from hematoxylin and eosin images. We further explored the feasibility of predicting TMB levels, i.e. high and low TMB, by utilizing a residual network (Resnet)-based deep learning algorithm for histopathological image analysis. Moreover, we developed a multimodal fusion deep learning model that combines histopathological images with omics data to predict TMB levels. We evaluated the performance of our models against various state-of-the-art methods using different TMB thresholds and obtained promising results. Specifically, our histopathological image analysis model achieved an area under curve (AUC) of 0.749. Notably, the multimodal fusion model significantly outperformed the model that relied only on histopathological images, with the highest AUC of 0.971. Our findings suggest that histopathological images could be used with reasonable accuracy to predict TMB levels in gastric cancer patients, while multimodal deep learning could achieve even higher levels of accuracy. This study sheds new light on predicting TMB in gastric cancer patients.

Predicting colorectal cancer tumor mutational burden from histopathological images and clinical information using multi-modal deep learning.

MOTIVATION: Tumor mutational burden (TMB) is an indicator of the efficacy and prognosis of immune checkpoint therapy in colorectal cancer (CRC). In general, patients with higher TMB values are more likely to benefit from immunotherapy. Though whole-exome sequencing is considered the gold standard for determining TMB, it is difficult to be applied in clinical practice due to its high cost. There are also a few DNA panel-based methods to estimate TMB; however, their detection cost is also high, and the associated wet-lab experiments usually take days, which emphasize the need for faster and cheaper alternatives. RESULTS: In this study, we propose a multi-modal deep learning model based on a residual network (ResNet) and multi-modal compact bilinear pooling to predict TMB status (i.e. TMB high (TMB_H) or TMB low(TMB_L)) directly from histopathological images and clinical data. We applied the model to CRC data from The Cancer Genome Atlas and compared it with four other popular methods, namely, ResNet18, ResNet50, VGG19 and AlexNet. We tested different TMB thresholds, namely, percentiles of 10%, 14.3%, 15%, 16.3%, 20%, 30% and 50%, to differentiate TMB_H and TMB_L.For the percentile of 14.3% (i.e. TMB value 20) and ResNet18, our model achieved an area under the receiver operating characteristic curve of 0.817 after 5-fold cross-validation, which was better than that of other compared models. In addition, we also found that TMB values were significantly associated with the tumor stage and N and M stages. Our study shows that deep learning models can predict TMB status from histopathological images and clinical information only, which is worth clinical application.

Prediction of Tumor Mutation Load in Colorectal Cancer Histopathological Images Based on Deep Learning.

Colorectal cancer (CRC) is one of the most prevalent malignancies, and immunotherapy can be applied to CRC patients of all ages, while its efficacy is uncertain. Tumor mutational burden (TMB) is important for predicting the effect of immunotherapy. Currently, whole-exome sequencing (WES) is a standard method to measure TMB, but it is costly and inefficient. Therefore, it is urgent to explore a method to assess TMB without WES to improve immunotherapy outcomes. In this study, we propose a deep learning method, DeepHE, based on the Residual Network (ResNet) model. On images of tissue, DeepHE can efficiently identify and analyze characteristics of tumor cells in CRC to predict the TMB. In our study, we used ×40 magnification images and grouped them by patients followed by thresholding at the 10th and 20th quantiles, which significantly improves the performance. Also, our model is superior compared with multiple models. In summary, deep learning methods can explore the association between histopathological images and genetic mutations, which will contribute to the precise treatment of CRC patients.

Predicting Tumor Mutational Burden From Lung Adenocarcinoma Histopathological Images Using Deep Learning.

Tumor mutation burden (TMB) is an important biomarker for tumor immunotherapy. It plays an important role in the clinical treatment process, but the gold standard measurement of TMB is based on whole exome sequencing (WES). WES cannot be done in most hospitals due to its high cost, long turnaround times and operational complexity. To seek out a better method to evaluate TMB, we divided the patients with lung adenocarcinoma (LUAD) in TCGA into two groups according to the TMB value, then analyzed the differences of clinical characteristics and gene expression between the two groups. We further explored the possibility of using histopathological images to predict TMB status, and developed a deep learning model to predict TMB based on histopathological images of LUAD. In the 5-fold cross-validation, the area under the receiver operating characteristic (ROC) curve (AUC) of the model was 0.64. This study showed that it is possible to use deep learning to predict genomic features from histopathological images, though the prediction accuracy was relatively low. The study opens up a new way to explore the relationship between genes and phenotypes.

Prediction of Target-Drug Therapy by Identifying Gene Mutations in Lung Cancer With Histopathological Stained Image and Deep Learning Techniques.

Lung cancer is a kind of cancer with high morbidity and mortality which is associated with various gene mutations. Individualized targeted-drug therapy has become the optimized treatment of lung cancer, especially benefit for patients who are not qualified for lung lobectomy. It is crucial to accurately identify mutant genes within tumor region from stained pathological slice. Therefore, we mainly focus on identifying mutant gene of lung cancer by analyzing the pathological images. In this study, we have proposed a method by identifying gene mutations in lung cancer with histopathological stained image and deep learning to predict target-drug therapy, referred to as DeepIMLH. The DeepIMLH algorithm first downloaded 180 hematoxylin-eosin staining (H&E) images of lung cancer from the Cancer Gene Atlas (TCGA). Then deep convolution Gaussian mixture model (DCGMM) was used to perform color normalization. Convolutional neural network (CNN) and residual network (Res-Net) were used to identifying mutated gene from H&E stained imaging and achieved good accuracy. It demonstrated that our method can be used to choose targeted-drug therapy which might be applied to clinical practice. More studies should be conducted though.

Identification and Validation of Efficacy of Immunological Therapy for Lung Cancer From Histopathological Images Based on Deep Learning.

Cancer immunotherapy, as a novel treatment against cancer metastasis and recurrence, has brought a significantly promising and effective therapy for cancer treatments. At present, programmed death 1 (PD-1) and programmed cell death-Ligand 1 (PD-L1) treatment for lung cancer is primarily recognized as an immune checkpoint inhibitor (ICI) to play an anti-tumor effect; however, it remains uncertain regarding of its efficacy though. Thereafter, tumor mutation burden (TMB) was recognized as a high-potential to be a predictive marker for the immune therapy, but it is invasive and costly. Therefore, discovering more immune-related biomarkers that have a guiding role in immunotherapy is a crucial step in the development of immunotherapy. In our study, we proposed a deep convolutional neural network (CNN)-based framework, DeepLRHE, which can efficiently analyze immunological stained pathological images of lung cancer tissues, as well as to identify and explore pathogenesis which can be used for immunological treatment in clinical field. In this study, we used 180 whole slice images (WSIs) of lung cancer downloaded from TCGA which was model training and validation. After two cross-validation used for this model, we compared with the area under the curve (AUC) of multiple mutant genes, TP53 had highest AUC, which reached 0.87, and EGFR, DNMT3A, PBRM1, STK11 also reached ranged from 0.71 to 0.84. The study results showed that the deep learning can used to assist health professionals for target-therapy as well as immunotherapies, therefore to improve the disease prognosis.