Clinically applicable deep learning-based decision aids for treatment of neovascular AMD.

PURPOSE: Anti-vascular endothelial growth factor (Anti-VEGF) therapy is currently seen as the standard for treatment of neovascular AMD (nAMD). However, while treatments are highly effective, decisions for initial treatment and retreatment are often challenging for non-retina specialists. The purpose of this study is to develop convolutional neural networks (CNN) that can differentiate treatment indicated presentations of nAMD for referral to treatment centre based solely on SD-OCT. This provides the basis for developing an applicable medical decision support system subsequently. METHODS: SD-OCT volumes of a consecutive real-life cohort of 1503 nAMD patients were analysed and two experiments were carried out. To differentiate between no treatment class vs. initial treatment nAMD class and stabilised nAMD vs. active nAMD, two novel CNNs, based on SD-OCT volume scans, were developed and tested for robustness and performance. In a step towards explainable artificial intelligence (AI), saliency maps of the SD-OCT volume scans of 24 initial indication decisions with a predicted probability of > 97.5% were analysed (score 0-2 in respect to staining intensity). An AI benchmark against retina specialists was performed. RESULTS: At the first experiment, the area under curve (AUC) of the receiver-operating characteristic (ROC) for the differentiation of patients for the initial analysis was 0.927 (standard deviation (SD): 0.018), for the second experiment (retreatment analysis) 0.865 (SD: 0.027). The results were robust to downsampling (» of the original resolution) and cross-validation (tenfold). In addition, there was a high correlation between the AI analysis and expert opinion in a sample of 102 cases for differentiation of patients needing treatment (κ = 0.824). On saliency maps, the relevant structures for individual initial indication decisions were the retina/vitreous interface, subretinal space, intraretinal cysts, subretinal pigment epithelium space, and the choroid. CONCLUSION: The developed AI algorithms can define and differentiate presentations of AMD, which should be referred for treatment or retreatment with anti-VEGF therapy. This may support non-retina specialists to interpret SD-OCT on expert opinion level. The individual decision of the algorithm can be supervised by saliency maps.

Deep learning-based assessment of body composition and liver tumour burden for survival modelling in advanced colorectal cancer.

BACKGROUND: Personalized therapy planning remains a significant challenge in advanced colorectal cancer care, despite extensive research on prognostic and predictive markers. A strong correlation of sarcopenia or overall body composition and survival has been described. Here, we explore whether automated assessment of body composition and liver metastases from standard of care CT images can add to clinical parameters in personalized survival risk prognostication. METHODS: We retrospectively analysed clinical imaging data from 85 patients (50.6% female, mean age 58.9 SD 12.2 years) with colorectal cancer and synchronous liver metastases. Pretrained deep learning models were used to assess body composition and liver metastasis geometry from abdominal CT images before the initiation of systemic treatment. Abdominal muscle-to-bone ratio (MBR) was calculated by dividing abdominal muscle volume by abdominal bone volume. MBR was compared with body mass index (BMI), abdominal muscle volume, and abdominal muscle volume divided by height squared. Differences in overall survival based on body composition and liver metastasis parameters were compared using Kaplan-Meier survival curves. Results were correlated with clinical and biomarker data to develop a machine learning model for survival risk prognostication. RESULTS: The MBR, unlike abdominal muscle volume or BMI, was significantly associated with overall survival (HR 0.39, 95% CI: 0.19-0.80, P = 0.009). The MBR (P = 0.022), liver metastasis surface area (P = 0.01) and primary tumour sidedness (P = 0.007) were independently associated with overall survival in multivariate analysis. Body composition parameters did not correlate with KRAS mutational status or primary tumour sidedness. A prediction model based on MBR, liver metastasis surface area and primary tumour sidedness achieved a concordance index of 0.69. CONCLUSIONS: Automated segmentation enables to extract prognostic parameters from routine imaging data for personalized survival modelling in advanced colorectal cancer patients.

Valuing vicinity: Memory attention framework for context-based semantic segmentation in histopathology.

The segmentation of histopathological whole slide images into tumourous and non-tumourous types of tissue is a challenging task that requires the consideration of both local and global spatial contexts to classify tumourous regions precisely. The identification of subtypes of tumour tissue complicates the issue as the sharpness of separation decreases and the pathologist's reasoning is even more guided by spatial context. However, the identification of detailed tissue types is crucial for providing personalized cancer therapies. Due to the high resolution of whole slide images, existing semantic segmentation methods, restricted to isolated image sections, are incapable of processing context information beyond. To take a step towards better context comprehension, we propose a patch neighbour attention mechanism to query the neighbouring tissue context from a patch embedding memory bank and infuse context embeddings into bottleneck hidden feature maps. Our memory attention framework (MAF) mimics a pathologist's annotation procedure - zooming out and considering surrounding tissue context. The framework can be integrated into any encoder-decoder segmentation method. We evaluate the MAF on two public breast cancer and liver cancer data sets and an internal kidney cancer data set using famous segmentation models (U-Net, DeeplabV3) and demonstrate the superiority over other context-integrating algorithms - achieving a substantial improvement of up to 17% on Dice score. The code is publicly available at https://github.com/tio-ikim/valuing-vicinity.