Heterogeneous Tumor-Immune Microenvironments among Differentially Growing Metastases in an Ovarian Cancer Patient.

We present an exceptional case of a patient with high-grade serous ovarian cancer, treated with multiple chemotherapy regimens, who exhibited regression of some metastatic lesions with concomitant progression of other lesions during a treatment-free period. Using immunogenomic approaches, we found that progressing metastases were characterized by immune cell exclusion, whereas regressing and stable metastases were infiltrated by CD8+ and CD4+ T cells and exhibited oligoclonal expansion of specific T cell subsets. We also detected CD8+ T cell reactivity against predicted neoepitopes after isolation of cells from a blood sample taken almost 3 years after the tumors were resected. These findings suggest that multiple distinct tumor immune microenvironments co-exist within a single individual and may explain in part the heterogeneous fates of metastatic lesions often observed in the clinic post-therapy. VIDEO ABSTRACT.

Assessing Endoscopic Response in Locally Advanced Rectal Cancer Treated with Total Neoadjuvant Therapy: Development and Validation of a Highly Accurate Convolutional Neural Network.

BACKGROUND: Rectal tumors display varying degrees of response to total neoadjuvant therapy (TNT). We evaluated the performance of a convolutional neural network (CNN) in interpreting endoscopic images of either a non-complete response to TNT or local regrowth during watch-and-wait surveillance. METHODS: Endoscopic images from stage II/III rectal cancers treated with TNT from 2012 to 2020 at a single institution were retrospectively reviewed. Images were labelled as Tumor or No Tumor based on endoscopy timing (before, during, or after treatment) and the tumor's endoluminal response. A CNN was trained using ResNet-50 architecture. The area under the curve (AUC) was analyzed during training and for two test sets. The main test set included images of tumors treated with TNT. The other contained images of local regrowth. The model's performance was compared to sixteen surgeons and surgical trainees who evaluated 119 images for evidence of tumor. Fleiss' kappa was calculated by respondent experience level. RESULTS: A total of 2717 images from 288 patients were included; 1407 (51.8%) contained tumor. The AUC was 0.99, 0.98, and 0.92 for training, main test, and local regrowth test sets. The model performed on par with surgeons of all experience levels for the main test set. Interobserver agreement was good ( k = 0.71-0.81). All groups outperformed the model in identifying tumor from images of local regrowth. Interobserver agreement was fair to moderate ( k = 0.24-0.52). CONCLUSIONS: A highly accurate CNN matched the performance of colorectal surgeons in identifying a noncomplete response to TNT. However, the model demonstrated suboptimal accuracy when analyzing images of local regrowth.

Deep Learning-Based Model for Identifying Tumors in Endoscopic Images From Patients With Locally Advanced Rectal Cancer Treated With Total Neoadjuvant Therapy.

BACKGROUND: A barrier to the widespread adoption of watch-and-wait management for locally advanced rectal cancer is the inaccuracy and variability of identifying tumor response endoscopically in patients who have completed total neoadjuvant therapy (chemoradiotherapy and systemic chemotherapy). OBJECTIVE: This study aimed to develop a novel method of identifying the presence or absence of a tumor in endoscopic images using deep convolutional neural network-based automatic classification and to assess the accuracy of the method. DESIGN: In this prospective pilot study, endoscopic images obtained before, during, and after total neoadjuvant therapy were grouped on the basis of tumor presence. A convolutional neural network was modified for probabilistic classification of tumor versus no tumor and trained with an endoscopic image set. After training, a testing endoscopic imaging set was applied to the network. SETTINGS: The study was conducted at a comprehensive cancer center. PATIENTS: Images were analyzed from 109 patients who were diagnosed with locally advanced rectal cancer between December 2012 and July 2017 and who underwent total neoadjuvant therapy. MAIN OUTCOME MEASURES: The main outcomes were accuracy of identifying tumor presence or absence in endoscopic images measured as area under the receiver operating characteristic for the training and testing image sets. RESULTS: A total of 1392 images were included; 1099 images (468 of no tumor and 631 of tumor) were for training and 293 images (151 of no tumor and 142 of tumor) for testing. The area under the receiver operating characteristic for training and testing was 0.83. LIMITATIONS: The study had a limited number of images in each set and was conducted at a single institution. CONCLUSIONS: The convolutional neural network method is moderately accurate in distinguishing tumor from no tumor. Further research should focus on validating the convolutional neural network on a large image set. See Video Abstract at http://links.lww.com/DCR/B959 . MODELO BASADO EN APRENDIZAJE PROFUNDO PARA IDENTIFICAR TUMORES EN IMGENES ENDOSCPICAS DE PACIENTES CON CNCER DE RECTO LOCALMENTE AVANZADO TRATADOS CON TERAPIA NEOADYUVANTE TOTAL: ANTECEDENTES:Una barrera para la aceptación generalizada del tratamiento de Observar y Esperar para el cáncer de recto localmente avanzado, es la imprecisión y la variabilidad en la identificación de la respuesta tumoral endoscópica, en pacientes que completaron la terapia neoadyuvante total (quimiorradioterapia y quimioterapia sistémica).OBJETIVO:Desarrollar un método novedoso para identificar la presencia o ausencia de un tumor en imágenes endoscópicas utilizando una clasificación automática basada en redes neuronales convolucionales profundas y evaluar la precisión del método.DISEÑO:Las imágenes endoscópicas obtenidas antes, durante y después de la terapia neoadyuvante total se agruparon en base de la presencia del tumor. Se modificó una red neuronal convolucional para la clasificación probabilística de tumor versus no tumor y se entrenó con un conjunto de imágenes endoscópicas. Después del entrenamiento, se aplicó a la red un conjunto de imágenes endoscópicas de prueba.ENTORNO CLINICO:El estudio se realizó en un centro oncológico integral.PACIENTES:Analizamos imágenes de 109 pacientes que fueron diagnosticados de cáncer de recto localmente avanzado entre diciembre de 2012 y julio de 2017 y que se sometieron a terapia neoadyuvante total.PRINCIPALES MEDIDAS DE VALORACION:La precisión en la identificación de la presencia o ausencia de tumores en imágenes endoscópicas medidas como el área bajo la curva de funcionamiento del receptor para los conjuntos de imágenes de entrenamiento y prueba.RESULTADOS:Se incluyeron mil trescientas noventa y dos imágenes: 1099 (468 sin tumor y 631 con tumor) para entrenamiento y 293 (151 sin tumor y 142 con tumor) para prueba. El área bajo la curva operativa del receptor para entrenamiento y prueba fue de 0,83.LIMITACIONES:El estudio tuvo un número limitado de imágenes en cada conjunto y se realizó en una sola institución.CONCLUSIÓN:El método de la red neuronal convolucional es moderadamente preciso para distinguir el tumor de ningún tumor. La investigación adicional debería centrarse en validar la red neuronal convolucional en un conjunto de imágenes mayor. Consulte Video Resumen en http://links.lww.com/DCR/B959 . (Traducción -Dr. Fidel Ruiz Healy ).

A machine learning model that classifies breast cancer pathologic complete response on MRI post-neoadjuvant chemotherapy.

BACKGROUND: For breast cancer patients undergoing neoadjuvant chemotherapy (NAC), pathologic complete response (pCR; no invasive or in situ) cannot be assessed non-invasively so all patients undergo surgery. The aim of our study was to develop and validate a radiomics classifier that classifies breast cancer pCR post-NAC on MRI prior to surgery. METHODS: This retrospective study included women treated with NAC for breast cancer from 2014 to 2016 with (1) pre- and post-NAC breast MRI and (2) post-NAC surgical pathology report assessing response. Automated radiomics analysis of pre- and post-NAC breast MRI involved image segmentation, radiomics feature extraction, feature pre-filtering, and classifier building through recursive feature elimination random forest (RFE-RF) machine learning. The RFE-RF classifier was trained with nested five-fold cross-validation using (a) radiomics only (model 1) and (b) radiomics and molecular subtype (model 2). Class imbalance was addressed using the synthetic minority oversampling technique. RESULTS: Two hundred seventy-three women with 278 invasive breast cancers were included; the training set consisted of 222 cancers (61 pCR, 161 no-pCR; mean age 51.8 years, SD 11.8), and the independent test set consisted of 56 cancers (13 pCR, 43 no-pCR; mean age 51.3 years, SD 11.8). There was no significant difference in pCR or molecular subtype between the training and test sets. Model 1 achieved a cross-validation AUROC of 0.72 (95% CI 0.64, 0.79) and a similarly accurate (P = 0.1) AUROC of 0.83 (95% CI 0.71, 0.94) in both the training and test sets. Model 2 achieved a cross-validation AUROC of 0.80 (95% CI 0.72, 0.87) and a similar (P = 0.9) AUROC of 0.78 (95% CI 0.62, 0.94) in both the training and test sets. CONCLUSIONS: This study validated a radiomics classifier combining radiomics with molecular subtypes that accurately classifies pCR on MRI post-NAC.

Integration of proteomics with CT-based qualitative and radiomic features in high-grade serous ovarian cancer patients: an exploratory analysis.

OBJECTIVES: To investigate the association between CT imaging traits and texture metrics with proteomic data in patients with high-grade serous ovarian cancer (HGSOC). METHODS: This retrospective, hypothesis-generating study included 20 patients with HGSOC prior to primary cytoreductive surgery. Two readers independently assessed the contrast-enhanced computed tomography (CT) images and extracted 33 imaging traits, with a third reader adjudicating in the event of a disagreement. In addition, all sites of suspected HGSOC were manually segmented texture features which were computed from each tumor site. Three texture features that represented intra- and inter-site tumor heterogeneity were used for analysis. An integrated analysis of transcriptomic and proteomic data identified proteins with conserved expression between primary tumor sites and metastasis. Correlations between protein abundance and various CT imaging traits and texture features were assessed using the Kendall tau rank correlation coefficient and the Mann-Whitney U test, whereas the area under the receiver operating characteristic curve (AUC) was reported as a metric of the strength and the direction of the association. P values < 0.05 were considered significant. RESULTS: Four proteins were associated with CT-based imaging traits, with the strongest correlation observed between the CRIP2 protein and disease in the mesentery (p < 0.001, AUC = 0.05). The abundance of three proteins was associated with texture features that represented intra-and inter-site tumor heterogeneity, with the strongest negative correlation between the CKB protein and cluster dissimilarity (p = 0.047, τ = 0.326). CONCLUSION: This study provides the first insights into the potential associations between standard-of-care CT imaging traits and texture measures of intra- and inter-site heterogeneity, and the abundance of several proteins. KEY POINTS: • CT-based texture features of intra- and inter-site tumor heterogeneity correlate with the abundance of several proteins in patients with HGSOC. • CT imaging traits correlate with protein abundance in patients with HGSOC.

Automated Breast Density Measurements From Chest Computed Tomography Scans.

To develop an automated method for quantifying percent breast density from chest computed tomography (CT) scans. A naïve Bayesian classifier based on gray-level intensities and spatial relationships was developed on CT scans from 10 patients diagnosed with Hodgkin lymphoma (HL) and imaged as part of routine clinical care. The algorithm was validated on CT scans from 75 additional HL patients. The classifier was developed and validated using a reference dataset with consensus manual segmentation of fibroglandular tissue. Accuracy was evaluated at the pixel-level to examine how well the algorithm identified pixels with fibroglandular tissue using true and false positive fractions (TPF and FPF, respectively). Quantitative estimates of the patient-level CT percent density were contrasted to each other using the concordance correlation coefficient, ρc, and to subjective ACR BI-RADS density assessments using Kendall's τb. The pixel-level TPF for identifying pixels with fibroglandular tissue was 82.7% (interquartile range of patient-specific TPFs 65.5%-89.6%). The pixel-level FPF was 9.2% (interquartile range of patient-specific FPFs 2.5%-45.3%). Patient-level agreement of the algorithm's automated density estimate with that obtained from the reference dataset was high, ρc = 0.93 (95% CI 0.90-0.96) as was agreement with a radiologist's subjective ACR-BI-RADS assessments, τb = 0.77. It is possible to obtain automated measurements of percent density from clinical CT scans.

MR Imaging of Rectal Cancer: Radiomics Analysis to Assess Treatment Response after Neoadjuvant Therapy.

Purpose To investigate the value of T2-weighted-based radiomics compared with qualitative assessment at T2-weighted imaging and diffusion-weighted (DW) imaging for diagnosis of clinical complete response in patients with rectal cancer after neoadjuvant chemotherapy-radiation therapy (CRT). Materials and Methods This retrospective study included 114 patients with rectal cancer who underwent magnetic resonance (MR) imaging after CRT between March 2012 and February 2016. Median age among women (47 of 114, 41%) was 55.9 years (interquartile range, 45.4-66.7 years) and median age among men (67 of 114, 59%) was 55 years (interquartile range, 48-67 years). Surgical histopathologic analysis was the reference standard for pathologic complete response (pCR). For qualitative assessment, two radiologists reached a consensus. For radiomics, one radiologist segmented the volume of interest on high-spatial-resolution T2-weighted images. A random forest classifier was trained to separate the patients by their outcomes after balancing the number of patients in each response category by using the synthetic minority oversampling technique. Statistical analysis was performed by using the Wilcoxon rank-sum test, McNemar test, and Benjamini-Hochberg method. Results Twenty-one of 114 patients (18%) achieved pCR. The radiomic classifier demonstrated an area under the curve of 0.93 (95% confidence interval [CI]: 0.87, 0.96), sensitivity of 100% (95% CI: 0.84, 1), specificity of 91% (95% CI: 0.84, 0.96), positive predictive value of 72% (95% CI: 0.53, 0.87), and negative predictive value of 100% (95% CI: 0.96, 1). The diagnostic performance of radiomics was significantly higher than was qualitative assessment at T2-weighted imaging or DW imaging alone (P < .02). The specificity and positive predictive values were significantly higher in radiomics than were at combined T2-weighted and DW imaging (P < .0001). Conclusion T2-weighted-based radiomics showed better classification performance compared with qualitative assessment at T2-weighted and DW imaging for diagnosing pCR in patients with locally advanced rectal cancer after CRT. © RSNA, 2018 Online supplemental material is available for this article.

Automatic classification of prostate cancer Gleason scores from multiparametric magnetic resonance images.

Noninvasive, radiological image-based detection and stratification of Gleason patterns can impact clinical outcomes, treatment selection, and the determination of disease status at diagnosis without subjecting patients to surgical biopsies. We present machine learning-based automatic classification of prostate cancer aggressiveness by combining apparent diffusion coefficient (ADC) and T2-weighted (T2-w) MRI-based texture features. Our approach achieved reasonably accurate classification of Gleason scores (GS) 6(3 + 3) vs. ≥7 and 7(3 + 4) vs. 7(4 + 3) despite the presence of highly unbalanced samples by using two different sample augmentation techniques followed by feature selection-based classification. Our method distinguished between GS 6(3 + 3) and ≥7 cancers with 93% accuracy for cancers occurring in both peripheral (PZ) and transition (TZ) zones and 92% for cancers occurring in the PZ alone. Our approach distinguished the GS 7(3 + 4) from GS 7(4 + 3) with 92% accuracy for cancers occurring in both the PZ and TZ and with 93% for cancers occurring in the PZ alone. In comparison, a classifier using only the ADC mean achieved a top accuracy of 58% for distinguishing GS 6(3 + 3) vs. GS ≥7 for cancers occurring in PZ and TZ and 63% for cancers occurring in PZ alone. The same classifier achieved an accuracy of 59% for distinguishing GS 7(3 + 4) from GS 7(4 + 3) occurring in the PZ and TZ and 60% for cancers occurring in PZ alone. Separate analysis of the cancers occurring in TZ alone was not performed owing to the limited number of samples. Our results suggest that texture features derived from ADC and T2-w MRI together with sample augmentation can help to obtain reasonably accurate classification of Gleason patterns.

Differentiation of Uterine Leiomyosarcoma from Atypical Leiomyoma: Diagnostic Accuracy of Qualitative MR Imaging Features and Feasibility of Texture Analysis.

PURPOSE: To investigate whether qualitative magnetic resonance (MR) features can distinguish leiomyosarcoma (LMS) from atypical leiomyoma (ALM) and assess the feasibility of texture analysis (TA). METHODS: This retrospective study included 41 women (ALM = 22, LMS = 19) imaged with MRI prior to surgery. Two readers (R1, R2) evaluated each lesion for qualitative MR features. Associations between MR features and LMS were evaluated with Fisher's exact test. Accuracy measures were calculated for the four most significant features. TA was performed for 24 patients (ALM = 14, LMS = 10) with uniform imaging following lesion segmentation on axial T2-weighted images. Texture features were pre-selected using Wilcoxon signed-rank test with Bonferroni correction and analyzed with unsupervised clustering to separate LMS from ALM. RESULTS: Four qualitative MR features most strongly associated with LMS were nodular borders, haemorrhage, "T2 dark" area(s), and central unenhanced area(s) (p ≤ 0.0001 each feature/reader). The highest sensitivity [1.00 (95%CI:0.82-1.00)/0.95 (95%CI: 0.74-1.00)] and specificity [0.95 (95%CI:0.77-1.00)/1.00 (95%CI:0.85-1.00)] were achieved for R1/R2, respectively, when a lesion had ≥3 of these four features. Sixteen texture features differed significantly between LMS and ALM (p-values: <0.001-0.036). Unsupervised clustering achieved accuracy of 0.75 (sensitivity: 0.70; specificity: 0.79). CONCLUSIONS: Combination of ≥3 qualitative MR features accurately distinguished LMS from ALM. TA was feasible. KEY POINTS: • Four qualitative MR features demonstrated the strongest statistical association with LMS. • Combination of ≥3 these features could accurately differentiate LMS from ALM. • Texture analysis was a feasible semi-automated approach for lesion categorization.

A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome.

PURPOSE: To evaluate the associations between clinical outcomes and radiomics-derived inter-site spatial heterogeneity metrics across multiple metastatic lesions on CT in patients with high-grade serous ovarian cancer (HGSOC). METHODS: IRB-approved retrospective study of 38 HGSOC patients. All sites of suspected HGSOC involvement on preoperative CT were manually segmented. Gray-level correlation matrix-based textures were computed from each tumour site, and grouped into five clusters using a Gaussian Mixture Model. Pairwise inter-site similarities were computed, generating an inter-site similarity matrix (ISM). Inter-site texture heterogeneity metrics were computed from the ISM and compared to clinical outcomes. RESULTS: Of the 12 inter-site texture heterogeneity metrics evaluated, those capturing the differences in texture similarities across sites were associated with shorter overall survival (inter-site similarity entropy, similarity level cluster shade, and inter-site similarity level cluster prominence; p ≤ 0.05) and incomplete surgical resection (similarity level cluster shade, inter-site similarity level cluster prominence and inter-site cluster variance; p ≤ 0.05). Neither the total number of disease sites per patient nor the overall tumour volume per patient was associated with overall survival. Amplification of 19q12 involving cyclin E1 gene (CCNE1) predominantly occurred in patients with more heterogeneous inter-site textures. CONCLUSION: Quantitative metrics non-invasively capturing spatial inter-site heterogeneity may predict outcomes in patients with HGSOC. KEY POINTS: • Calculating inter-site texture-based heterogeneity metrics was feasible • Metrics capturing texture similarities across HGSOC sites were associated with overall survival • Heterogeneity metrics were also associated with incomplete surgical resection of HGSOC.

Breast cancer molecular subtype classifier that incorporates MRI features.

PURPOSE: To use features extracted from magnetic resonance (MR) images and a machine-learning method to assist in differentiating breast cancer molecular subtypes. MATERIALS AND METHODS: This retrospective Health Insurance Portability and Accountability Act (HIPAA)-compliant study received Institutional Review Board (IRB) approval. We identified 178 breast cancer patients between 2006-2011 with: 1) ERPR + (n = 95, 53.4%), ERPR-/HER2 + (n = 35, 19.6%), or triple negative (TN, n = 48, 27.0%) invasive ductal carcinoma (IDC), and 2) preoperative breast MRI at 1.5T or 3.0T. Shape, texture, and histogram-based features were extracted from each tumor contoured on pre- and three postcontrast MR images using in-house software. Clinical and pathologic features were also collected. Machine-learning-based (support vector machines) models were used to identify significant imaging features and to build models that predict IDC subtype. Leave-one-out cross-validation (LOOCV) was used to avoid model overfitting. Statistical significance was determined using the Kruskal-Wallis test. RESULTS: Each support vector machine fit in the LOOCV process generated a model with varying features. Eleven out of the top 20 ranked features were significantly different between IDC subtypes with P < 0.05. When the top nine pathologic and imaging features were incorporated, the predictive model distinguished IDC subtypes with an overall accuracy on LOOCV of 83.4%. The combined pathologic and imaging model's accuracy for each subtype was 89.2% (ERPR+), 63.6% (ERPR-/HER2+), and 82.5% (TN). When only the top nine imaging features were incorporated, the predictive model distinguished IDC subtypes with an overall accuracy on LOOCV of 71.2%. The combined pathologic and imaging model's accuracy for each subtype was 69.9% (ERPR+), 62.9% (ERPR-/HER2+), and 81.0% (TN). CONCLUSION: We developed a machine-learning-based predictive model using features extracted from MRI that can distinguish IDC subtypes with significant predictive power. J. Magn. Reson. Imaging 2016;44:122-129.

Breast cancer subtype intertumor heterogeneity: MRI-based features predict results of a genomic assay.

PURPOSE: To investigate the association between a validated, gene-expression-based, aggressiveness assay, Oncotype Dx RS, and morphological and texture-based image features extracted from magnetic resonance imaging (MRI). MATERIALS AND METHODS: This retrospective study received Internal Review Board approval and need for informed consent was waived. Between 2006-2012, we identified breast cancer patients with: 1) ER+, PR+, and HER2- invasive ductal carcinoma (IDC); 2) preoperative breast MRI; and 3) Oncotype Dx RS test results. Extracted features included morphological, histogram, and gray-scale correlation matrix (GLCM)-based texture features computed from tumors contoured on pre- and three postcontrast MR images. Linear regression analysis was performed to investigate the association between Oncotype Dx RS and different clinical, pathologic, and imaging features. P < 0.05 was considered statistically significant. RESULTS: Ninety-five patients with IDC were included with a median Oncotype Dx RS of 16 (range: 0-45). Using stepwise multiple linear regression modeling, two MR-derived image features, kurtosis in the first and third postcontrast images and histologic nuclear grade, were found to be significantly correlated with the Oncotype Dx RS with P = 0.0056, 0.0005, and 0.0105, respectively. The overall model resulted in statistically significant correlation with Oncotype Dx RS with an R-squared value of 0.23 (adjusted R-squared = 0.20; P = 0.0002) and a Spearman's rank correlation coefficient of 0.49 (P < 0.0001). CONCLUSION: A model for IDC using imaging and pathology information correlates with Oncotype Dx RS scores, suggesting that image-based features could also predict the likelihood of recurrence and magnitude of chemotherapy benefit.

Haralick texture analysis of prostate MRI: utility for differentiating non-cancerous prostate from prostate cancer and differentiating prostate cancers with different Gleason scores.

OBJECTIVES: To investigate Haralick texture analysis of prostate MRI for cancer detection and differentiating Gleason scores (GS). METHODS: One hundred and forty-seven patients underwent T2- weighted (T2WI) and diffusion-weighted prostate MRI. Cancers ≥0.5 ml and non-cancerous peripheral (PZ) and transition (TZ) zone tissue were identified on T2WI and apparent diffusion coefficient (ADC) maps, using whole-mount pathology as reference. Texture features (Energy, Entropy, Correlation, Homogeneity, Inertia) were extracted and analysed using generalized estimating equations. RESULTS: PZ cancers (n = 143) showed higher Entropy and Inertia and lower Energy, Correlation and Homogeneity compared to non-cancerous tissue on T2WI and ADC maps (p-values: <.0001-0.008). In TZ cancers (n = 43) we observed significant differences for all five texture features on the ADC map (all p-values: <.0001) and for Correlation (p = 0.041) and Inertia (p = 0.001) on T2WI. On ADC maps, GS was associated with higher Entropy (GS 6 vs. 7: p = 0.0225; 6 vs. >7: p = 0.0069) and lower Energy (GS 6 vs. 7: p = 0.0116, 6 vs. >7: p = 0.0039). ADC map Energy (p = 0.0102) and Entropy (p = 0.0019) were significantly different in GS ≤3 + 4 versus ≥4 + 3 cancers; ADC map Entropy remained significant after controlling for the median ADC (p = 0.0291). CONCLUSION: Several Haralick-based texture features appear useful for prostate cancer detection and GS assessment. KEY POINTS: • Several Haralick texture features may differentiate non-cancerous and cancerous prostate tissue. • Tumour Energy and Entropy on ADC maps correlate with Gleason score. • T2w-image-derived texture features are not associated with the Gleason score.

Wasserstein HOG: Local Directionality Extraction via Optimal Transport.

Directionally sensitive radiomic features including the histogram of oriented gradient (HOG) have been shown to provide objective and quantitative measures for predicting disease outcomes in multiple cancers. However, radiomic features are sensitive to imaging variabilities including acquisition differences, imaging artifacts and noise, making them impractical for using in the clinic to inform patient care. We treat the problem of extracting robust local directionality features by mapping via optimal transport a given local image patch to an iso-intense patch of its mean. We decompose the transport map into sub-work costs each transporting in different directions. To test our approach, we evaluated the ability of the proposed approach to quantify tumor heterogeneity from magnetic resonance imaging (MRI) scans of brain glioblastoma multiforme, computed tomography (CT) scans of head and neck squamous cell carcinoma as well as longitudinal CT scans in lung cancer patients treated with immunotherapy. By considering the entropy difference of the extracted local directionality within tumor regions, we found that patients with higher entropy in their images, had significantly worse overall survival for all three datasets, which indicates that tumors that have images exhibiting flows in many directions may be more malignant. This may seem to reflect high tumor histologic grade or disorganization. Furthermore, by comparing the changes in entropy longitudinally using two imaging time points, we found patients with reduction in entropy from baseline CT are associated with longer overall survival (hazard ratio = 1.95, 95% confidence interval of 1.4-2.8, p = 1.65e-5). The proposed method provides a robust, training free approach to quantify the local directionality contained in images.

Artificial intelligence-based automated segmentation and radiotherapy dose mapping for thoracic normal tissues.

Background and purpose: Objective assessment of delivered radiotherapy (RT) to thoracic organs requires fast and accurate deformable dose mapping. The aim of this study was to implement and evaluate an artificial intelligence (AI) deformable image registration (DIR) and organ segmentation-based AI dose mapping (AIDA) applied to the esophagus and the heart. Materials and methods: AIDA metrics were calculated for 72 locally advanced non-small cell lung cancer patients treated with concurrent chemo-RT to 60 Gy in 2 Gy fractions in an automated pipeline. The pipeline steps were: (i) automated rigid alignment and cropping of planning CT to week 1 and week 2 cone-beam CT (CBCT) field-of-views, (ii) AI segmentation on CBCTs, and (iii) AI-DIR-based dose mapping to compute dose metrics. AIDA dose metrics were compared to the planned dose and manual contour dose mapping (manual DA). Results: AIDA required âˆ¼2 min/patient. Esophagus and heart segmentations were generated with a mean Dice similarity coefficient (DSC) of 0.80±0.15 and 0.94±0.05, a Hausdorff distance at 95th percentile (HD95) of 3.9±3.4 mm and 14.1±8.3 mm, respectively. AIDA heart dose was significantly lower than the planned heart dose (p = 0.04). Larger dose deviations (>=1Gy) were more frequently observed between AIDA and the planned dose (N = 26) than with manual DA (N = 6). Conclusions: Rapid estimation of RT dose to thoracic tissues from CBCT is feasible with AIDA. AIDA-derived metrics and segmentations were similar to manual DA, thus motivating the use of AIDA for RT applications.

Auto-segmentation of neck nodal metastases using self-distilled masked image transformer on longitudinal MR images.

Objectives: Auto-segmentation promises greater speed and lower inter-reader variability than manual segmentations in radiation oncology clinical practice. This study aims to implement and evaluate the accuracy of the auto-segmentation algorithm, "Masked Image modeling using the vision Transformers (SMIT)," for neck nodal metastases on longitudinal T2-weighted (T2w) MR images in oropharyngeal squamous cell carcinoma (OPSCC) patients. Methods: This prospective clinical trial study included 123 human papillomaviruses (HPV-positive [+]) related OSPCC patients who received concurrent chemoradiotherapy. T2w MR images were acquired on 3 T at pre-treatment (Tx), week 0, and intra-Tx weeks (1-3). Manual delineations of metastatic neck nodes from 123 OPSCC patients were used for the SMIT auto-segmentation, and total tumor volumes were calculated. Standard statistical analyses compared contour volumes from SMIT vs manual segmentation (Wilcoxon signed-rank test [WSRT]), and Spearman's rank correlation coefficients (ρ) were computed. Segmentation accuracy was evaluated on the test data set using the dice similarity coefficient (DSC) metric value. P-values <0.05 were considered significant. Results: No significant difference in manual and SMIT delineated tumor volume at pre-Tx (8.68 ± 7.15 vs 8.38 ± 7.01 cm3, P = 0.26 [WSRT]), and the Bland-Altman method established the limits of agreement as -1.71 to 2.31 cm3, with a mean difference of 0.30 cm3. SMIT model and manually delineated tumor volume estimates were highly correlated (ρ = 0.84-0.96, P < 0.001). The mean DSC metric values were 0.86, 0.85, 0.77, and 0.79 at the pre-Tx and intra-Tx weeks (1-3), respectively. Conclusions: The SMIT algorithm provides sufficient segmentation accuracy for oncological applications in HPV+ OPSCC. Advances in knowledge: First evaluation of auto-segmentation with SMIT using longitudinal T2w MRI in HPV+ OPSCC.

A model for gastrointestinal tract motility in a 4D imaging phantom of human anatomy.

BACKGROUND: Gastrointestinal (GI) tract motility is one of the main sources for intra/inter-fraction variability and uncertainty in radiation therapy for abdominal targets. Models for GI motility can improve the assessment of delivered dose and contribute to the development, testing, and validation of deformable image registration (DIR) and dose-accumulation algorithms. PURPOSE: To implement GI tract motion in the 4D extended cardiac-torso (XCAT) digital phantom of human anatomy. MATERIALS AND METHODS: Motility modes that exhibit large amplitude changes in the diameter of the GI tract and may persist over timescales comparable to online adaptive planning and radiotherapy delivery were identified based on literature research. Search criteria included amplitude changes larger than planning risk volume expansions and durations of the order of tens of minutes. The following modes were identified: peristalsis, rhythmic segmentation, high amplitude propagating contractions (HAPCs), and tonic contractions. Peristalsis and rhythmic segmentations were modeled by traveling and standing sinusoidal waves. HAPCs and tonic contractions were modeled by traveling and stationary Gaussian waves. Wave dispersion in the temporal and spatial domain was implemented by linear, exponential, and inverse power law functions. Modeling functions were applied to the control points of the nonuniform rational B-spline surfaces defined in the reference XCAT library. GI motility was combined with the cardiac and respiratory motions available in the standard 4D-XCAT phantom. Default model parameters were estimated based on the analysis of cine MRI acquisitions in 10 patients treated in a 1.5T MR-linac. RESULTS: We demonstrate the ability to generate realistic 4D multimodal images that simulate GI motility combined with respiratory and cardiac motion. All modes of motility, except tonic contractions, were observed in the analysis of our cine MRI acquisitions. Peristalsis was the most common. Default parameters estimated from cine MRI were used as initial values for simulation experiments. It is shown that in patients undergoing stereotactic body radiotherapy for abdominal targets, the effects of GI motility can be comparable or larger than the effects of respiratory motion. CONCLUSION: The digital phantom provides realistic models to aid in medical imaging and radiation therapy research. The addition of GI motility will further contribute to the development, testing, and validation of DIR and dose accumulation algorithms for MR-guided radiotherapy.

Deep learning-based dominant index lesion segmentation for MR-guided radiation therapy of prostate cancer.

BACKGROUND: Dose escalation radiotherapy enables increased control of prostate cancer (PCa) but requires segmentation of dominant index lesions (DIL). This motivates the development of automated methods for fast, accurate, and consistent segmentation of PCa DIL. PURPOSE: To construct and validate a model for deep-learning-based automatic segmentation of PCa DIL defined by Gleason score (GS) ≥3+4 from MR images applied to MR-guided radiation therapy. Validate generalizability of constructed models across scanner and acquisition differences. METHODS: Five deep-learning networks were evaluated on apparent diffusion coefficient (ADC) MRI from 500 lesions in 365 patients arising from internal training Dataset 1 (156 lesions in 125 patients, 1.5Tesla GE MR with endorectal coil), testing using Dataset 1 (35 lesions in 26 patients), external ProstateX Dataset 2 (299 lesions in 204 patients, 3Tesla Siemens MR), and internal inter-rater Dataset 3 (10 lesions in 10 patients, 3Tesla Philips MR). The five networks include: multiple resolution residually connected network (MRRN) and MRRN regularized in training with deep supervision implemented into the last convolutional block (MRRN-DS), Unet, Unet++, ResUnet, and fast panoptic segmentation (FPSnet) as well as fast panoptic segmentation with smoothed labels (FPSnet-SL). Models were evaluated by volumetric DIL segmentation accuracy using Dice similarity coefficient (DSC) and the balanced F1 measure of detection accuracy, as a function of lesion aggressiveness and size (Dataset 1 and 2), and accuracy with respect to two-raters (on Dataset 3). Upon acceptance for publication segmentation models will be made available in an open-source GitHub repository. RESULTS: In general, MRRN-DS more accurately segmented tumors than other methods on the testing datasets. MRRN-DS significantly outperformed ResUnet in Dataset2 (DSC of 0.54 vs. 0.44, p < 0.001) and the Unet++ in Dataset3 (DSC of 0.45 vs. p = 0.04). FPSnet-SL was similarly accurate as MRRN-DS in Dataset2 (p = 0.30), but MRRN-DS significantly outperformed FPSnet and FPSnet-SL in both Dataset1 (0.60 vs. 0.51 [p = 0.01] and 0.54 [p = 0.049] respectively) and Dataset3 (0.45 vs. 0.06 [p = 0.002] and 0.24 [p = 0.004] respectively). Finally, MRRN-DS produced slightly higher agreement with experienced radiologist than two radiologists in Dataset 3 (DSC of 0.45 vs. 0.41). CONCLUSIONS: MRRN-DS was generalizable to different MR testing datasets acquired using different scanners. It produced slightly higher agreement with an experienced radiologist than that between two radiologists. Finally, MRRN-DS more accurately segmented aggressive lesions, which are generally candidates for radiative dose ablation.

Progressively refined deep joint registration segmentation (ProRSeg) of gastrointestinal organs at risk: Application to MRI and cone-beam CT.

BACKGROUND: Adaptive radiation treatment (ART) for locally advanced pancreatic cancer (LAPC) requires consistently accurate segmentation of the extremely mobile gastrointestinal (GI) organs at risk (OAR) including the stomach, duodenum, large and small bowel. Also, due to lack of sufficiently accurate and fast deformable image registration (DIR), accumulated dose to the GI OARs is currently only approximated, further limiting the ability to more precisely adapt treatments. PURPOSE: Develop a 3-D Progressively refined joint Registration-Segmentation (ProRSeg) deep network to deformably align and segment treatment fraction magnetic resonance images (MRI)s, then evaluate segmentation accuracy, registration consistency, and feasibility for OAR dose accumulation. METHOD: ProRSeg was trained using five-fold cross-validation with 110 T2-weighted MRI acquired at five treatment fractions from 10 different patients, taking care that same patient scans were not placed in training and testing folds. Segmentation accuracy was measured using Dice similarity coefficient (DSC) and Hausdorff distance at 95th percentile (HD95). Registration consistency was measured using coefficient of variation (CV) in displacement of OARs. Statistical comparison to other deep learning and iterative registration methods were done using the Kruskal-Wallis test, followed by pair-wise comparisons with Bonferroni correction applied for multiple testing. Ablation tests and accuracy comparisons against multiple methods were done. Finally, applicability of ProRSeg to segment cone-beam CT (CBCT) scans was evaluated on a publicly available dataset of 80 scans using five-fold cross-validation. RESULTS: ProRSeg processed 3D volumes (128 × 192 × 128) in 3 s on a NVIDIA Tesla V100 GPU. It's segmentations were significantly more accurate ( p < 0.001 $p<0.001$ ) than compared methods, achieving a DSC of 0.94 ±0.02 for liver, 0.88±0.04 for large bowel, 0.78±0.03 for small bowel and 0.82±0.04 for stomach-duodenum from MRI. ProRSeg achieved a DSC of 0.72±0.01 for small bowel and 0.76±0.03 for stomach-duodenum from public CBCT dataset. ProRSeg registrations resulted in the lowest CV in displacement (stomach-duodenum C V x $CV_{x}$ : 0.75%, C V y $CV_{y}$ : 0.73%, and C V z $CV_{z}$ : 0.81%; small bowel C V x $CV_{x}$ : 0.80%, C V y $CV_{y}$ : 0.80%, and C V z $CV_{z}$ : 0.68%; large bowel C V x $CV_{x}$ : 0.71%, C V y $CV_{y}$ : 0.81%, and C V z $CV_{z}$ : 0.75%). ProRSeg based dose accumulation accounting for intra-fraction (pre-treatment to post-treatment MRI scan) and inter-fraction motion showed that the organ dose constraints were violated in four patients for stomach-duodenum and for three patients for small bowel. Study limitations include lack of independent testing and ground truth phantom datasets to measure dose accumulation accuracy. CONCLUSIONS: ProRSeg produced more accurate and consistent GI OARs segmentation and DIR of MRI and CBCTs compared to multiple methods. Preliminary results indicates feasibility for OAR dose accumulation using ProRSeg.

Artificial Intelligence in CT and MR Imaging for Oncological Applications.

Cancer care increasingly relies on imaging for patient management. The two most common cross-sectional imaging modalities in oncology are computed tomography (CT) and magnetic resonance imaging (MRI), which provide high-resolution anatomic and physiological imaging. Herewith is a summary of recent applications of rapidly advancing artificial intelligence (AI) in CT and MRI oncological imaging that addresses the benefits and challenges of the resultant opportunities with examples. Major challenges remain, such as how best to integrate AI developments into clinical radiology practice, the vigorous assessment of quantitative CT and MR imaging data accuracy, and reliability for clinical utility and research integrity in oncology. Such challenges necessitate an evaluation of the robustness of imaging biomarkers to be included in AI developments, a culture of data sharing, and the cooperation of knowledgeable academics with vendor scientists and companies operating in radiology and oncology fields. Herein, we will illustrate a few challenges and solutions of these efforts using novel methods for synthesizing different contrast modality images, auto-segmentation, and image reconstruction with examples from lung CT as well as abdome, pelvis, and head and neck MRI. The imaging community must embrace the need for quantitative CT and MRI metrics beyond lesion size measurement. AI methods for the extraction and longitudinal tracking of imaging metrics from registered lesions and understanding the tumor environment will be invaluable for interpreting disease status and treatment efficacy. This is an exciting time to work together to move the imaging field forward with narrow AI-specific tasks. New AI developments using CT and MRI datasets will be used to improve the personalized management of cancer patients.