Deep Learning on Multimodal Chemical and Whole Slide Imaging Data for Predicting Prostate Cancer Directly from Tissue Images.

Prostate cancer is one of the most common cancers globally and is the second most common cancer in the male population in the US. Here we develop a study based on correlating the hematoxylin and eosin (H&E)-stained biopsy data with MALDI mass-spectrometric imaging data of the corresponding tissue to determine the cancerous regions and their unique chemical signatures and variations of the predicted regions with original pathological annotations. We obtain features from high-resolution optical micrographs of whole slide H&E stained data through deep learning and spatially register them with mass spectrometry imaging (MSI) data to correlate the chemical signature with the tissue anatomy of the data. We then use the learned correlation to predict prostate cancer from observed H&E images using trained coregistered MSI data. This multimodal approach can predict cancerous regions with ∼80% accuracy, which indicates a correlation between optical H&E features and chemical information found in MSI. We show that such paired multimodal data can be used for training feature extraction networks on H&E data which bypasses the need to acquire expensive MSI data and eliminates the need for manual annotation saving valuable time. Two chemical biomarkers were also found to be predicting the ground truth cancerous regions. This study shows promise in generating improved patient treatment trajectories by predicting prostate cancer directly from readily available H&E-stained biopsy images aided by coregistered MSI data.

Classifying cancer pathology reports with hierarchical self-attention networks.

We introduce a deep learning architecture, hierarchical self-attention networks (HiSANs), designed for classifying pathology reports and show how its unique architecture leads to a new state-of-the-art in accuracy, faster training, and clear interpretability. We evaluate performance on a corpus of 374,899 pathology reports obtained from the National Cancer Institute's (NCI) Surveillance, Epidemiology, and End Results (SEER) program. Each pathology report is associated with five clinical classification tasks - site, laterality, behavior, histology, and grade. We compare the performance of the HiSAN against other machine learning and deep learning approaches commonly used on medical text data - Naive Bayes, logistic regression, convolutional neural networks, and hierarchical attention networks (the previous state-of-the-art). We show that HiSANs are superior to other machine learning and deep learning text classifiers in both accuracy and macro F-score across all five classification tasks. Compared to the previous state-of-the-art, hierarchical attention networks, HiSANs not only are an order of magnitude faster to train, but also achieve about 1% better relative accuracy and 5% better relative macro F-score.

Deep Transfer Learning Across Cancer Registries for Information Extraction from Pathology Reports.

Automated text information extraction from cancer pathology reports is an active area of research to support national cancer surveillance. A well-known challenge is how to develop information extraction tools with robust performance across cancer registries. In this study we investigated whether transfer learning (TL) with a convolutional neural network (CNN) can facilitate cross-registry knowledge sharing. Specifically, we performed a series of experiments to determine whether a CNN trained with single-registry data is capable of transferring knowledge to another registry or whether developing a cross-registry knowledge database produces a more effective and generalizable model. Using data from two cancer registries and primary tumor site and topography as the information extraction task of interest, our study showed that TL results in 6.90% and 17.22% improvement of classification macro F-score over the baseline single-registry models. Detailed analysis illustrated that the observed improvement is evident in the low prevalence classes.

Investigating the feasibility of rapid MRI for image-guided motion management in lung cancer radiotherapy.

Cycle-to-cycle variations in respiratory motion can cause significant geometric and dosimetric errors in the administration of lung cancer radiation therapy. A common limitation of the current strategies for motion management is that they assume a constant, reproducible respiratory cycle. In this work, we investigate the feasibility of using rapid MRI for providing long-term imaging of the thorax in order to better capture cycle-to-cycle variations. Two nonsmall-cell lung cancer patients were imaged (free-breathing, no extrinsic contrast, and 1.5 T scanner). A balanced steady-state-free-precession (b-SSFP) sequence was used to acquire cine-2D and cine-3D (4D) images. In the case of Patient 1 (right midlobe lesion, ~40 mm diameter), tumor motion was well correlated with diaphragmatic motion. In the case of Patient 2, (left upper-lobe lesion, ~60 mm diameter), tumor motion was poorly correlated with diaphragmatic motion. Furthermore, the motion of the tumor centroid was poorly correlated with the motion of individual points on the tumor boundary, indicating significant rotation and/or deformation. These studies indicate that image quality and acquisition speed of cine-2D MRI were adequate for motion monitoring. However, significant improvements are required to achieve comparable speeds for truly 4D MRI. Despite several challenges, rapid MRI offers a feasible and attractive tool for noninvasive, long-term motion monitoring.

Dosimetric evaluation of a "virtual" image-guidance alternative to explicit 6 degree of freedom robotic couch correction.

PURPOSE: Clinical evaluation of a "virtual" methodology for providing 6 degrees of freedom (6DOF) patient set-up corrections and comparison to corrections facilitated by a 6DOF robotic couch. METHODS: A total of 55 weekly in-room image-guidance computed tomographic (CT) scans were acquired using a CT-on-rails for 11 pelvic and head and neck cancer patients treated at our facility. Fusion of the CT-of-the-day to the simulation CT allowed prototype virtual 6DOF correction software to calculate the translations, single couch yaw, and beam-specific gantry and collimator rotations necessary to effectively reproduce the same corrections as a 6DOF robotic couch. These corrections were then used to modify the original treatment plan beam geometry and this modified plan geometry was applied to the CT-of-the-day to evaluate the dosimetric effects of the virtual correction method. This virtual correction dosimetry was compared with calculated geometric and dosimetric results for an explicit 6DOF robotic couch correction methodology. RESULTS: A (2%, 2mm) gamma analysis comparing dose distributions created using the virtual corrections to those from explicit corrections showed that an average of 95.1% of all points had a gamma of 1 or less, with a standard deviation of 3.4%. For a total of 470 dosimetric metrics (ie, maximum and mean dose statistics for all relevant structures) compared for all 55 image-guidance sessions, the average dose difference for these metrics between the plans employing the virtual corrections and the explicit corrections was -0.12% with a standard deviation of 0.82%; 97.9% of all metrics were within 2%. CONCLUSIONS: Results showed that the virtual corrections yielded dosimetric distributions that were essentially equivalent to those obtained when 6DOF robotic corrections were used, and that always outperformed the most commonly employed clinical approach of 3 translations only. This suggests that for the patient datasets studied here, highly effective image-guidance corrections can be made without the use of a robotic couch.