Multi-scale relational graph convolutional network for multiple instance learning in histopathology images.

Graph convolutional neural networks have shown significant potential in natural and histopathology images. However, their use has only been studied in a single magnification or multi-magnification with either homogeneous graphs or only different node types. In order to leverage the multi-magnification information and improve message passing with graph convolutional networks, we handle different embedding spaces at each magnification by introducing the Multi-Scale Relational Graph Convolutional Network (MS-RGCN) as a multiple instance learning method. We model histopathology image patches and their relation with neighboring patches and patches at other scales (i.e., magnifications) as a graph. We define separate message-passing neural networks based on node and edge types to pass the information between different magnification embedding spaces. We experiment on prostate cancer histopathology images to predict the grade groups based on the extracted features from patches. We also compare our MS-RGCN with multiple state-of-the-art methods with evaluations on several source and held-out datasets. Our method outperforms the state-of-the-art on all of the datasets and image types consisting of tissue microarrays, whole-mount slide regions, and whole-slide images. Through an ablation study, we test and show the value of the pertinent design features of the MS-RGCN.

Prostate Cancer Risk Stratification by Digital Histopathology and Deep Learning.

PURPOSE: Prostate cancer (PCa) represents a highly heterogeneous disease that requires tools to assess oncologic risk and guide patient management and treatment planning. Current models are based on various clinical and pathologic parameters including Gleason grading, which suffers from a high interobserver variability. In this study, we determine whether objective machine learning (ML)-driven histopathology image analysis would aid us in better risk stratification of PCa. MATERIALS AND METHODS: We propose a deep learning, histopathology image-based risk stratification model that combines clinicopathologic data along with hematoxylin and eosin- and Ki-67-stained histopathology images. We train and test our model, using a five-fold cross-validation strategy, on a data set from 502 treatment-naïve PCa patients who underwent radical prostatectomy (RP) between 2000 and 2012. RESULTS: We used the concordance index as a measure to evaluate the performance of various risk stratification models. Our risk stratification model on the basis of convolutional neural networks demonstrated superior performance compared with Gleason grading and the Cancer of the Prostate Risk Assessment Post-Surgical risk stratification models. Using our model, 3.9% of the low-risk patients were correctly reclassified to be high-risk and 21.3% of the high-risk patients were correctly reclassified as low-risk. CONCLUSION: These findings highlight the importance of ML as an objective tool for histopathology image assessment and patient risk stratification. With further validation on large cohorts, the digital pathology risk classification we propose may be helpful in guiding administration of adjuvant therapy including radiotherapy after RP.