Recurrent Oropharyngeal Squamous Cell Carcinomas Maintain Anti-tumor Immunity and Multinucleation Levels Following Completion of Radiation.

OBJECTIVE: Oropharyngeal squamous cell carcinoma (OPSCC) recurrence is almost universally fatal. Development of effective therapeutic options requires an improved understanding of recurrent OPSCC biology. METHODS: We analyzed paired primary-recurrent OPSCC from Veterans treated at the Michael E. DeBakey Veterans Affairs Medical Center between 2000 and 2020 who received curative intent radiation-based treatment (with or without chemotherapy). Patient tumors were analyzed using standard immunohistochemistry and automated imaging of infiltrating lymphocytes and multinucleated tumor cells coupled to machine learning algorithms. RESULTS: Primary and recurrent tumors demonstrated high concordance via p16 and p53 immunohistochemistry, with comparable levels of multinucleation. In contrast, recurrent tumors demonstrated significantly higher levels of CD8+ tumor infiltrating lymphocytes (p<0.05) and higher levels of PD-L1 expression (p<0.05). CONCLUSION: Exposure to chemo-radiation and recurrence following treatment preserves critical features of intrinsic tumor biology and the tumor immune microenvironment suggesting that novel treatment regimens may be as effective in the salvage setting as in the definitive intent setting.

Visual Assessment of 2-Dimensional Levels Within 3-Dimensional Pathology Data Sets of Prostate Needle Biopsies Reveals Substantial Spatial Heterogeneity.

Prostate cancer prognostication largely relies on visual assessment of a few thinly sectioned biopsy specimens under a microscope to assign a Gleason grade group (GG). Unfortunately, the assigned GG is not always associated with a patient's outcome in part because of the limited sampling of spatially heterogeneous tumors achieved by 2-dimensional histopathology. In this study, open-top light-sheet microscopy was used to obtain 3-dimensional pathology data sets that were assessed by 4 human readers. Intrabiopsy variability was assessed by asking readers to perform Gleason grading of 5 different levels per biopsy for a total of 20 core needle biopsies (ie, 100 total images). Intrabiopsy variability (Cohen κ) was calculated as the worst pairwise agreement in GG between individual levels within each biopsy and found to be 0.34, 0.34, 0.38, and 0.43 for the 4 pathologists. These preliminary results reveal that even within a 1-mm-diameter needle core, GG based on 2-dimensional images can vary dramatically depending on the location within a biopsy being analyzed. We believe that morphologic assessment of whole biopsies in 3 dimension has the potential to enable more reliable and consistent tumor grading.

Recurrent oropharyngeal squamous cell carcinomas maintain anti-tumor immunity and multinucleation levels following completion of radiation.

Objective: Oropharyngeal squamous cell carcinoma (OPSCC) recurrence is almost universally fatal. Development of effective therapeutic options requires an improved understanding of recurrent OPSCC biology. Methods: We analyzed paired primary-recurrent OPSCC from Veterans treated at the Michael E. DeBakey Veterans Affairs Medical Center between 2000 and 2020 who received curative intent radiation-based treatment (with or without chemotherapy). Patient tumors were analyzed using standard immunohistochemistry and automated imaging of infiltrating lymphocytes and multinucleated tumor cells coupled to machine learning algorithms. Results: Primary and recurrent tumors demonstrated high concordance via p16 and p53 immunohistochemistry, with comparable levels of multinucleation. In contrast, recurrent tumors demonstrated significantly higher levels of CD8+ tumor infiltrating lymphocytes (p<0.05) and higher levels of PD-L1 expression (p<0.05). Conclusion: Exposure to chemo-radiation and recurrence following treatment does not appear deleterious to underlying biological characteristics and anti-tumor immunity of oropharyngeal cancer, suggesting that novel treatment regimens may be as effective in the salvage setting as in the definitive intent setting.

Machine learning driven index of tumor multinucleation correlates with survival and suppressed anti-tumor immunity in head and neck squamous cell carcinoma patients.

OBJECTIVES: Matching treatment intensity to tumor biology is critical to precision oncology for head and neck squamous cell carcinoma (HNSCC) patients. We sought to identify biological features of tumor cell multinucleation, previously shown by us to correlate with survival in oropharyngeal (OP) SCC using a machine learning approach. MATERIALS AND METHODS: Hematoxylin and eosin images from an institutional OPSCC cohort formed the training set (DTr). TCGA HNSCC patients (oral cavity, oropharynx and larynx/hypopharynx) formed the validation set (DV). Deep learning models were trained in DTr to calculate a multinucleation index (MuNI) score. Gene set enrichment analysis (GSEA) was then used to explore correlations between MuNI and tumor biology. RESULTS: MuNI correlated with overall survival. A multivariable nomogram that included MuNI, age, race, sex, T/N stage, and smoking status yielded a C-index of 0.65, and MuNI was prognostic of overall survival (2.25, 1.07-4.71, 0.03), independent of the other variables. High MuNI scores correlated with depletion of effector immunocyte subsets across all HNSCC sites independent of HPV and TP53 mutational status although the correlations were strongest in wild-type TP53 tumors potentially due to aberrant mitotic events and activation of DNA-repair mechanisms. CONCLUSION: MuNI is associated with survival in HNSCC across subsites. This may be driven by an association between high levels of multinucleation and a suppressive (potentially exhausted) tumor immune microenvironment. Mechanistic studies examining the link between multinucleation and tumor immunity will be required to characterize biological drivers of multinucleation and their impact on treatment response and outcomes.

Computational pathology improves risk stratification of a multi-gene assay for early stage ER+ breast cancer.

Prognostic markers currently utilized in clinical practice for estrogen receptor-positive (ER+) and lymph node-negative (LN-) invasive breast cancer (IBC) patients include the Nottingham grading system and Oncotype Dx (ODx). However, these biomarkers are not always optimal and remain subject to inter-/intra-observer variability and high cost. In this study, we evaluated the association between computationally derived image features from H&E images and disease-free survival (DFS) in ER+ and LN- IBC. H&E images from a total of n = 321 patients with ER+ and LN- IBC from three cohorts were employed for this study (Training set: D1 (n = 116), Validation sets: D2 (n = 121) and D3 (n = 84)). A total of 343 features relating to nuclear morphology, mitotic activity, and tubule formation were computationally extracted from each slide image. A Cox regression model (IbRiS) was trained to identify significant predictors of DFS and predict a high/low-risk category using D1 and was validated on independent testing sets D2 and D3 as well as within each ODx risk category. IbRiS was significantly prognostic of DFS with a hazard ratio (HR) of 2.33 (95% confidence interval (95% CI) = 1.02-5.32, p = 0.045) on D2 and a HR of 2.94 (95% CI = 1.18-7.35, p = 0.0208) on D3. In addition, IbRiS yielded significant risk stratification within high ODx risk categories (D1 + D2: HR = 10.35, 95% CI = 1.20-89.18, p = 0.0106; D1: p = 0.0238; D2: p = 0.0389), potentially providing more granular risk stratification than offered by ODx alone.

Nondestructive 3D pathology with analysis of nuclear features for prostate cancer risk assessment.

Prostate cancer treatment decisions rely heavily on subjective visual interpretation [assigning Gleason patterns or International Society of Urological Pathology (ISUP) grade groups] of limited numbers of two-dimensional (2D) histology sections. Under this paradigm, interobserver variance is high, with ISUP grades not correlating well with outcome for individual patients, and this contributes to the over- and undertreatment of patients. Recent studies have demonstrated improved prognostication of prostate cancer outcomes based on computational analyses of glands and nuclei within 2D whole slide images. Our group has also shown that the computational analysis of three-dimensional (3D) glandular features, extracted from 3D pathology datasets of whole intact biopsies, can allow for improved recurrence prediction compared to corresponding 2D features. Here we seek to expand on these prior studies by exploring the prognostic value of 3D shape-based nuclear features in prostate cancer (e.g. nuclear size, sphericity). 3D pathology datasets were generated using open-top light-sheet (OTLS) microscopy of 102 cancer-containing biopsies extracted ex vivo from the prostatectomy specimens of 46 patients. A deep learning-based workflow was developed for 3D nuclear segmentation within the glandular epithelium versus stromal regions of the biopsies. 3D shape-based nuclear features were extracted, and a nested cross-validation scheme was used to train a supervised machine classifier based on 5-year biochemical recurrence (BCR) outcomes. Nuclear features of the glandular epithelium were found to be more prognostic than stromal cell nuclear features (area under the ROC curve [AUC] = 0.72 versus 0.63). 3D shape-based nuclear features of the glandular epithelium were also more strongly associated with the risk of BCR than analogous 2D features (AUC = 0.72 versus 0.62). The results of this preliminary investigation suggest that 3D shape-based nuclear features are associated with prostate cancer aggressiveness and could be of value for the development of decision-support tools. © 2023 The Pathological Society of Great Britain and Ireland.

Multi-site cross-organ calibrated deep learning (MuSClD): Automated diagnosis of non-melanoma skin cancer.

Although deep learning (DL) has demonstrated impressive diagnostic performance for a variety of computational pathology tasks, this performance often markedly deteriorates on whole slide images (WSI) generated at external test sites. This phenomenon is due in part to domain shift, wherein differences in test-site pre-analytical variables (e.g., slide scanner, staining procedure) result in WSI with notably different visual presentations compared to training data. To ameliorate pre-analytic variances, approaches such as CycleGAN can be used to calibrate visual properties of images between sites, with the intent of improving DL classifier generalizability. In this work, we present a new approach termed Multi-Site Cross-Organ Calibration based Deep Learning (MuSClD) that employs WSIs of an off-target organ for calibration created at the same site as the on-target organ, based off the assumption that cross-organ slides are subjected to a common set of pre-analytical sources of variance. We demonstrate that by using an off-target organ from the test site to calibrate training data, the domain shift between training and testing data can be mitigated. Importantly, this strategy uniquely guards against potential data leakage introduced during calibration, wherein information only available in the testing data is imparted on the training data. We evaluate MuSClD in the context of the automated diagnosis of non-melanoma skin cancer (NMSC). Specifically, we evaluated MuSClD for identifying and distinguishing (a) basal cell carcinoma (BCC), (b) in-situ squamous cell carcinomas (SCC-In Situ), and (c) invasive squamous cell carcinomas (SCC-Invasive), using an Australian (training, n = 85) and a Swiss (held-out testing, n = 352) cohort. Our experiments reveal that MuSCID reduces the Wasserstein distances between sites in terms of color, contrast, and brightness metrics, without imparting noticeable artifacts to training data. The NMSC-subtyping performance is statistically improved as a result of MuSCID in terms of one-vs. rest AUC: BCC (0.92 vs 0.87, p = 0.01), SCC-In Situ (0.87 vs 0.73, p = 0.15) and SCC-Invasive (0.92 vs 0.82, p = 1e-5). Compared to baseline NMSC-subtyping with no calibration, the internal validation results of MuSClD (BCC (0.98), SCC-In Situ (0.92), and SCC-Invasive (0.97)) suggest that while domain shift indeed degrades classification performance, our on-target calibration using off-target tissue can safely compensate for pre-analytical variabilities, while improving the robustness of the model.

Image analysis reveals differences in tumor multinucleations in Black and White patients with human papillomavirus-associated oropharyngeal squamous cell carcinoma.

BACKGROUND: Understanding biological differences between different racial groups of human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma (OPSCC) patients, who have differences in terms of incidence, survival, and tumor morphology, can facilitate accurate prognostic biomarkers, which can help develop personalized treatment strategies. METHODS: This study evaluated whether there were morphologic differences between HPV-associated tumors from Black and White patients in terms of multinucleation index (MuNI), an image analysis-derived metric that measures density of multinucleated tumor cells within epithelial regions on hematoxylin-eosin images and previously has been prognostic in HPV-associated OPSCC patients. In this study, the authors specifically evaluated whether the same MuNI cutoff that was prognostic of overall survival (OS) and disease-free survival in their previous study, TTR , is valid for Black and White patients, separately. We also evaluated population-specific cutoffs, TB for Blacks and TW for Whites, for risk stratification. RESULTS: MuNI was statistically significantly different between Black (mean, 3.88e-4; median, 3.67e-04) and White patients (mean, 3.36e-04; median, 2.99e-04), with p = .0078. Using TTR , MuNI was prognostic of OS in the entire population with hazard ratio (HR) of 1.71 (p = .002; 95% confidence interval [CI], 1.21-2.43) and in White patients with HR of 1.72 (p = .005; 95% CI, 1.18-2.51). Population-specific cutoff, TW , yielded improved HR of 1.77 (p = .003; 95% CI, 1.21-2.58) for White patients, whereas TB did not improve risk-stratification in Black patients with HR of 0.6 (p = .3; HR, 0.6; 95% CI, 0.2-1.80). CONCLUSIONS: Histological difference between White and Black patient tumors in terms of multinucleated tumor cells suggests the need for considering population-specific prognostic biomarkers for personalized risk stratification strategies for HPV-associated OPSCC patients.

Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors.

Immune checkpoint inhibitors (ICIs) show prominent clinical activity across multiple advanced tumors. However, less than half of patients respond even after molecule-based selection. Thus, improved biomarkers are required. In this study, we use an image analysis to capture morphologic attributes relating to the spatial interaction and architecture of tumor cells and tumor-infiltrating lymphocytes (TILs) from digitized H&E images. We evaluate the association of image features with progression-free (PFS) and overall survival in non-small cell lung cancer (NSCLC) (N = 187) and gynecological cancer (N = 39) patients treated with ICIs. We demonstrated that the classifier trained with NSCLC alone was associated with PFS in independent NSCLC cohorts and also in gynecological cancer. The classifier was also associated with clinical outcome independent of clinical factors. Moreover, the classifier was associated with PFS even with low PD-L1 expression. These findings suggest that image analysis can be used to predict clinical end points in patients receiving ICI.

A machine learning model for separating epithelial and stromal regions in oral cavity squamous cell carcinomas using H&E-stained histology images: A multi-center, retrospective study.

OBJECTIVE: Tissue slides from Oral cavity squamous cell carcinoma (OC-SCC), particularly the epithelial regions, hold morphologic features that are both diagnostic and prognostic. Yet, previously developed approaches for automated epithelium segmentation in OC-SCC have not been independently tested in a multi-center setting. In this study, we aimed to investigate the effectiveness and applicability of a convolutional neural network (CNN) model to perform epithelial segmentation using digitized H&E-stained diagnostic slides from OC-SCC patients in a multi-center setting. METHODS: A CNN model was developed to segment the epithelial regions of digitized slides (n = 810), retrospectively collected from five different centers. Deep learning models were trained and validated using well-annotated tissue microarray (TMA) images (n = 212) at various magnifications. The best performing model was locked down and used for independent testing with a total of 478 whole-slide images (WSIs). Manually annotated epithelial regions were used as the reference standard for evaluation. We also compared the model generated results with IHC-stained epithelium (n = 120) as the reference. RESULTS: The locked-down CNN model trained on the TMA image training cohorts with 10x magnification achieved the best segmentation performance. The locked-down model performed consistently and yielded Pixel Accuracy, Recall Rate, Precision Rate, and Dice Coefficient that ranged from 95.8% to 96.6%, 79.1% to 93.8%, 85.7% to 89.3%, and 82.3% to 89.0%, respectively for the three independent testing WSI cohorts. CONCLUSION: The automated model achieved a consistently accurate performance for automated epithelial region segmentation compared to manual annotations. This model could be integrated into a computer-aided diagnosis or prognosis system.

Oropharyngeal cancer outcomes correlate with p16 status, multinucleation and immune infiltration.

Oropharyngeal squamous cell carcinoma (OPSCC), largely fueled by the human papillomavirus (HPV), has a complex biological and immunologic phenotype. Although HPV/p16 status can be used to stratify OPSCC patients as a function of survival, it remains unclear what drives an improved treatment response in HPV-associated OPSCC and whether targetable biomarkers exist that can inform a precision oncology approach. We analyzed OPSCC patients treated between 2000 and 2016 and correlated locoregional control (LRC), disease-free survival (DFS) and overall survival (OS) with conventional clinical parameters, risk parameters generated using deep-learning algorithms trained to quantify tumor-infiltrating lymphocytes (TILs) (OP-TIL) and multinucleated tumor cells (MuNI) and targeted transcriptomics. P16 was a dominant determinant of LRC, DFS and OS, but tobacco exposure, OP-TIL and MuNI risk features correlated with clinical outcomes independent of p16 status and the combination of p16, OP-TIL and MuNI generated a better stratification of OPSCC risk compared to individual parameters. Differential gene expression (DEG) analysis demonstrated overlap between MuNI and OP-TIL and identified genes involved in DNA repair, oxidative stress response and tumor immunity as the most prominent correlates with survival. Alteration of inflammatory/immune pathways correlated strongly with all risk features and oncologic outcomes. This suggests that development of OPSCC consists of an intersection between multiple required and permissive oncogenic and immunologic events which may be mechanistically linked. The strong relationship between tumor immunity and oncologic outcomes in OPSCC regardless of HPV status may provide opportunities for further biomarker development and precision oncology approaches incorporating immune checkpoint inhibitors for maximal anti-tumor efficacy.

Prostate Cancer Risk Stratification via Nondestructive 3D Pathology with Deep Learning-Assisted Gland Analysis.

Prostate cancer treatment planning is largely dependent upon examination of core-needle biopsies. The microscopic architecture of the prostate glands forms the basis for prognostic grading by pathologists. Interpretation of these convoluted three-dimensional (3D) glandular structures via visual inspection of a limited number of two-dimensional (2D) histology sections is often unreliable, which contributes to the under- and overtreatment of patients. To improve risk assessment and treatment decisions, we have developed a workflow for nondestructive 3D pathology and computational analysis of whole prostate biopsies labeled with a rapid and inexpensive fluorescent analogue of standard hematoxylin and eosin (H&E) staining. This analysis is based on interpretable glandular features and is facilitated by the development of image translation-assisted segmentation in 3D (ITAS3D). ITAS3D is a generalizable deep learning-based strategy that enables tissue microstructures to be volumetrically segmented in an annotation-free and objective (biomarker-based) manner without requiring immunolabeling. As a preliminary demonstration of the translational value of a computational 3D versus a computational 2D pathology approach, we imaged 300 ex vivo biopsies extracted from 50 archived radical prostatectomy specimens, of which, 118 biopsies contained cancer. The 3D glandular features in cancer biopsies were superior to corresponding 2D features for risk stratification of patients with low- to intermediate-risk prostate cancer based on their clinical biochemical recurrence outcomes. The results of this study support the use of computational 3D pathology for guiding the clinical management of prostate cancer. SIGNIFICANCE: An end-to-end pipeline for deep learning-assisted computational 3D histology analysis of whole prostate biopsies shows that nondestructive 3D pathology has the potential to enable superior prognostic stratification of patients with prostate cancer.

An Imaging Biomarker of Tumor-Infiltrating Lymphocytes to Risk-Stratify Patients With HPV-Associated Oropharyngeal Cancer.

BACKGROUND: Human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma (OPSCC) has excellent control rates compared to nonvirally associated OPSCC. Multiple trials are actively testing whether de-escalation of treatment intensity for these patients can maintain oncologic equipoise while reducing treatment-related toxicity. We have developed OP-TIL, a biomarker that characterizes the spatial interplay between tumor-infiltrating lymphocytes (TILs) and surrounding cells in histology images. Herein, we sought to test whether OP-TIL can segregate stage I HPV-associated OPSCC patients into low-risk and high-risk groups and aid in patient selection for de-escalation clinical trials. METHODS: Association between OP-TIL and patient outcome was explored on whole slide hematoxylin and eosin images from 439 stage I HPV-associated OPSCC patients across 6 institutional cohorts. One institutional cohort (n = 94) was used to identify the most prognostic features and train a Cox regression model to predict risk of recurrence and death. Survival analysis was used to validate the algorithm as a biomarker of recurrence or death in the remaining 5 cohorts (n = 345). All statistical tests were 2-sided. RESULTS: OP-TIL separated stage I HPV-associated OPSCC patients with 30 or less pack-year smoking history into low-risk (2-year disease-free survival [DFS] = 94.2%; 5-year DFS = 88.4%) and high-risk (2-year DFS = 82.5%; 5-year DFS = 74.2%) groups (hazard ratio = 2.56, 95% confidence interval = 1.52 to 4.32; P < .001), even after adjusting for age, smoking status, T and N classification, and treatment modality on multivariate analysis for DFS (hazard ratio = 2.27, 95% confidence interval = 1.32 to 3.94; P = .003). CONCLUSIONS: OP-TIL can identify stage I HPV-associated OPSCC patients likely to be poor candidates for treatment de-escalation. Following validation on previously completed multi-institutional clinical trials, OP-TIL has the potential to be a biomarker, beyond clinical stage and HPV status, that can be used clinically to optimize patient selection for de-escalation.

Radiomic Features Associated With HPV Status on Pretreatment Computed Tomography in Oropharyngeal Squamous Cell Carcinoma Inform Clinical Prognosis.

PURPOSE: There is a lack of biomarkers for accurately prognosticating outcome in both human papillomavirus-related (HPV+) and tobacco- and alcohol-related (HPV-) oropharyngeal squamous cell carcinoma (OPSCC). The aims of this study were to i) develop and evaluate radiomic features within (intratumoral) and around tumor (peritumoral) on CT scans to predict HPV status; ii) investigate the prognostic value of the radiomic features for both HPV- and HPV+ patients, including within individual AJCC eighth edition-defined stage groups; and iii) develop and evaluate a clinicopathologic imaging nomogram involving radiomic, clinical, and pathologic factors for disease-free survival (DFS) prediction for HPV+ patients. EXPERIMENTAL DESIGN: This retrospective study included 582 OPSCC patients, of which 462 were obtained from The Cancer Imaging Archive (TCIA) with available tumor segmentation and 120 were from Cleveland Clinic Foundation (CCF, denoted as SCCF) with HPV+ OPSCC. We subdivided the TCIA cohort into training (ST, 180 patients) and validation (SV, 282 patients) based on an approximately 3:5 ratio for HPV status prediction. The top 15 radiomic features that were associated with HPV status were selected by the minimum redundancy-maximum relevance (MRMR) using ST and evaluated on SV. Using 3 of these 15 top HPV status-associated features, we created radiomic risk scores for both HPV+ (RRSHPV+) and HPV- patients (RRSHPV-) through a Cox regression model to predict DFS. RRSHPV+ was further externally validated on SCCF. Nomograms for the HPV+ population (Mp+RRS) were constructed. Both RRSHPV+ and Mp+RRS were used to prognosticate DFS for the AJCC eighth edition-defined stage I, stage II, and stage III patients separately. RESULTS: RRSHPV+ was prognostic for DFS for i) the whole HPV+ population [hazard ratio (HR) = 1.97, 95% confidence interval (CI): 1.35-2.88, p < 0.001], ii) the AJCC eighth stage I population (HR = 1.99, 95% CI: 1.04-3.83, p = 0.039), and iii) the AJCC eighth stage II population (HR = 3.61, 95% CI: 1.71-7.62, p < 0.001). HPV+ nomogram Mp+RRS (C-index, 0.59; 95% CI: 0.54-0.65) was also prognostic of DFS (HR = 1.86, 95% CI: 1.27-2.71, p = 0.001). CONCLUSION: CT-based radiomic signatures are associated with both HPV status and DFS in OPSCC patients. With additional validation, the radiomic signature and its corresponding nomogram could potentially be used for identifying HPV+ OPSCC patients who might be candidates for therapy deintensification.

Computerized tumor multinucleation index (MuNI) is prognostic in p16+ oropharyngeal carcinoma.

BACKGROUNDPatients with p16+ oropharyngeal squamous cell carcinoma (OPSCC) are potentially cured with definitive treatment. However, there are currently no reliable biomarkers of treatment failure for p16+ OPSCC. Pathologist-based visual assessment of tumor cell multinucleation (MN) has been shown to be independently prognostic of disease-free survival (DFS) in p16+ OPSCC. However, its quantification is time intensive, subjective, and at risk of interobserver variability.METHODSWe present a deep-learning-based metric, the multinucleation index (MuNI), for prognostication in p16+ OPSCC. This approach quantifies tumor MN from digitally scanned H&E-stained slides. Representative H&E-stained whole-slide images from 1094 patients with previously untreated p16+ OPSCC were acquired from 6 institutions for optimization and validation of the MuNI.RESULTSThe MuNI was prognostic for DFS, overall survival (OS), or distant metastasis-free survival (DMFS) in p16+ OPSCC, with HRs of 1.78 (95% CI: 1.37-2.30), 1.94 (1.44-2.60), and 1.88 (1.43-2.47), respectively, independent of age, smoking status, treatment type, or tumor and lymph node (T/N) categories in multivariable analyses. The MuNI was also prognostic for DFS, OS, and DMFS in patients with stage I and stage III OPSCC, separately.CONCLUSIONMuNI holds promise as a low-cost, tissue-nondestructive, H&E stain-based digital biomarker test for counseling, treatment, and surveillance of patients with p16+ OPSCC. These data support further confirmation of the MuNI in prospective trials.FUNDINGNational Cancer Institute (NCI), NIH; National Institute for Biomedical Imaging and Bioengineering, NIH; National Center for Research Resources, NIH; VA Merit Review Award from the US Department of VA Biomedical Laboratory Research and Development Service; US Department of Defense (DOD) Breast Cancer Research Program Breakthrough Level 1 Award; DOD Prostate Cancer Idea Development Award; DOD Lung Cancer Investigator-Initiated Translational Research Award; DOD Peer-Reviewed Cancer Research Program; Ohio Third Frontier Technology Validation Fund; Wallace H. Coulter Foundation Program in the Department of Biomedical Engineering; Clinical and Translational Science Award (CTSA) program, Case Western Reserve University; NCI Cancer Center Support Grant, NIH; Career Development Award from the US Department of VA Clinical Sciences Research and Development Program; Dan L. Duncan Comprehensive Cancer Center Support Grant, NIH; and Computational Genomic Epidemiology of Cancer Program, Case Comprehensive Cancer Center. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH, the US Department of VA, the DOD, or the US Government.

Feature-driven local cell graph (FLocK): New computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers.

Local spatial arrangement of nuclei in histopathology images of different cancer subtypes has been shown to have prognostic value. In order to capture localized nuclear architectural information, local cell cluster graph-based measurements have been proposed. However, conventional ways of cell graph construction only utilize nuclear spatial proximity, and do not differentiate between different cell types while constructing the graph. In this paper, we present feature-driven local cell cluster graph (FLocK), a new approach to constructing local cell graphs by simultaneously considering spatial proximity and attributes of the individual nuclei (e.g. shape, size, texture). In addition, we have designed a new set of quantitative graph-derived metrics to be extracted from FLocKs, in turn capturing the interplay between different proximally located clusters of nuclei. We have evaluated the efficacy of FLocK features extracted from H&E stained tissue images in two clinical applications: to classify short-term vs. long-term survival among patients of early stage non-small cell lung cancer (ES-NSCLC), and also to predict human papillomavirus (HPV) status of oropharyngeal squamous cell carcinoma (OP-SCCs). In the classification of long-term vs. short-term survival among patients of ES-NSCLC (training cohort, n = 434), the top 10 discriminative FLocK features related to the variation of FLocK size and intersected FLocK distance were identified, via Minimum Redundancy and Maximum Relevance (MRMR) selection, in 100 runs of 10-fold cross-validation, and in conjunction with a linear discriminant classifier yielded a mean AUC of 0.68 for predicting survival in the training cohort. This is better than other state-of-art histomorphometric and deep learning classifiers (cell cluster graphs (AUC = 0.62), global cell graph (AUC = 0.56), nuclear shape (AUC = 0.54), nuclear orientation (AUC = 0.61), AlexNet (AUC = 0.55), ResNet (AUC = 0.56)). The FLocK-based classifier yielded an AUC of 0.70 in an independent testing cohort (n = 150). The patients identified as "high-risk" had significantly poorer overall survival in the testing cohort, with a hazard ratio (95% confidence interval) of 2.24 (1.24-4.05), p = 0.01144). In the classification of HPV status of OP-SCC, the top three FLocK features pertaining to the portion of intersected FLocKs were used to construct a classifier, which yielded an AUC of 0.80 in the training cohort (n = 50), and an accuracy of 0.78 in an independent testing cohort (n = 35). The combination of FLocK measurements with cell cluster graphs, nuclear orientation, and nuclear shape improved the training AUC to 0.87, 0.91 and 0.85, respectively. Deep learning approaches yielded marginally better performance than the FLocK-based classifier in this application, with AUC = 0.78 for AlexNet, AUC = 0.81 for ResNet, and AUC = 0.76 for FLocK-based classifier in the testing cohort. However, the combination of two hand-crafted features: FLocK and nuclear orientation yielded a better performance (AUC = 0.84). FLocK provides a unique and quantitative way to analyze histology images of solid tumors and interrogate tumor morphology from a different aspect than existing histomorphometrics. The source code can be accessed at https://github.com/hacylu/FLocK.

DeepDistance: A multi-task deep regression model for cell detection in inverted microscopy images.

This paper presents a new deep regression model, which we call DeepDistance, for cell detection in images acquired with inverted microscopy. This model considers cell detection as a task of finding most probable locations that suggest cell centers in an image. It represents this main task with a regression task of learning an inner distance metric. However, different than the previously reported regression based methods, the DeepDistance model proposes to approach its learning as a multi-task regression problem where multiple tasks are learned by using shared feature representations. To this end, it defines a secondary metric, normalized outer distance, to represent a different aspect of the problem and proposes to define its learning as complementary to the main cell detection task. In order to learn these two complementary tasks more effectively, the DeepDistance model designs a fully convolutional network (FCN) with a shared encoder path and end-to-end trains this FCN to concurrently learn the tasks in parallel. For further performance improvement on the main task, this paper also presents an extended version of the DeepDistance model that includes an auxiliary classification task and learns it in parallel to the two regression tasks by also sharing feature representations with them. DeepDistance uses the inner distances estimated by these FCNs in a detection algorithm to locate individual cells in a given image. In addition to this detection algorithm, this paper also suggests a cell segmentation algorithm that employs the estimated maps to find cell boundaries. Our experiments on three different human cell lines reveal that the proposed multi-task learning models, the DeepDistance model and its extended version, successfully identify the locations of cell as well as delineate their boundaries, even for the cell line that was not used in training, and improve the results of its counterparts.

Object-Oriented Segmentation of Cell Nuclei in Fluorescence Microscopy Images.

Cell nucleus segmentation remains an open and challenging problem especially to segment nuclei in cell clumps. Splitting a cell clump would be straightforward if the gradients of boundary pixels in-between the nuclei were always higher than the others. However, imperfections may exist: inhomogeneities of pixel intensities in a nucleus may cause to define spurious boundaries whereas insufficient pixel intensity differences at the border of overlapping nuclei may cause to miss some true boundary pixels. In contrast, these imperfections are typically observed at the pixel-level, causing local changes in pixel values without changing the semantics on a large scale. In response to these issues, this article introduces a new nucleus segmentation method that relies on using gradient information not at the pixel level but at the object level. To this end, it proposes to decompose an image into smaller homogeneous subregions, define edge-objects at four different orientations to encode the gradient information at the object level, and devise a merging algorithm, in which the edge-objects vote for subregion pairs along their orientations and the pairs are iteratively merged if they get sufficient votes from multiple orientations. Our experiments on fluorescence microscopy images reveal that this high-level representation and the design of a merging algorithm using edge-objects (gradients at the object level) improve the segmentation results.

Iterative h-minima-based marker-controlled watershed for cell nucleus segmentation.

Automated microscopy imaging systems facilitate high-throughput screening in molecular cellular biology research. The first step of these systems is cell nucleus segmentation, which has a great impact on the success of the overall system. The marker-controlled watershed is a technique commonly used by the previous studies for nucleus segmentation. These studies define their markers finding regional minima on the intensity/gradient and/or distance transform maps. They typically use the h-minima transform beforehand to suppress noise on these maps. The selection of the h value is critical; unnecessarily small values do not sufficiently suppress the noise, resulting in false and oversegmented markers, and unnecessarily large ones suppress too many pixels, causing missing and undersegmented markers. Because cell nuclei show different characteristics within an image, the same h value may not work to define correct markers for all the nuclei. To address this issue, in this work, we propose a new watershed algorithm that iteratively identifies its markers, considering a set of different h values. In each iteration, the proposed algorithm defines a set of candidates using a particular h value and selects the markers from those candidates provided that they fulfill the size requirement. Working with widefield fluorescence microscopy images, our experiments reveal that the use of multiple h values in our iterative algorithm leads to better segmentation results, compared to its counterparts. © 2016 International Society for Advancement of Cytometry.

Two-tier tissue decomposition for histopathological image representation and classification.

In digital pathology, devising effective image representations is crucial to design robust automated diagnosis systems. To this end, many studies have proposed to develop object-based representations, instead of directly using image pixels, since a histopathological image may contain a considerable amount of noise typically at the pixel-level. These previous studies mostly employ color information to define their objects, which approximately represent histological tissue components in an image, and then use the spatial distribution of these objects for image representation and classification. Thus, object definition has a direct effect on the way of representing the image, which in turn affects classification accuracies. In this paper, our aim is to design a classification system for histopathological images. Towards this end, we present a new model for effective representation of these images that will be used by the classification system. The contributions of this model are twofold. First, it introduces a new two-tier tissue decomposition method for defining a set of multityped objects in an image. Different than the previous studies, these objects are defined combining texture, shape, and size information and they may correspond to individual histological tissue components as well as local tissue subregions of different characteristics. As its second contribution, it defines a new metric, which we call dominant blob scale, to characterize the shape and size of an object with a single scalar value. Our experiments on colon tissue images reveal that this new object definition and characterization provides distinguishing representation of normal and cancerous histopathological images, which is effective to obtain more accurate classification results compared to its counterparts.