Evaluation of a task specific self-supervised learning framework in digital pathology relative to transfer learning approaches and existing foundation models.

An integral stage in typical digital pathology workflows involves deriving specific features from tiles extracted from a tessellated whole slide image. Notably, various computer vision neural network architectures, particularly the ImageNet pre-trained, have been extensively used in this domain. This study critically analyzes multiple strategies for encoding tiles to understand the extent of transfer learning and identify the most effective approach. The study categorizes neural network performance into three weight initialization methods: random, ImageNet-based, and self-supervised learning. Additionally, we propose a framework based on task-specific self-supervised learning (TS-SSL) which introduces a shallow feature extraction method, employing a spatial-channel attention block to glean distinctive features optimized for histopathology intricacies. Across two different downstream classification tasks (patch classification, and weakly supervised whole slide image classification) with diverse classification datasets, including Colorectal cancer histology, Patch Camelyon, PANDA, TCGA and CIFAR-10, our task specific self-supervised encoding approach consistently outperforms other CNN-based encoders. The better performances highlight the potential of task-specific-attention based self-supervised training in tailoring feature extraction for histopathology, indicating a shift from utilizing pretrained models originating outside the histopathology domain. Our study supports the idea that task-specific self-supervised learning allows domain-specific feature extraction, encouraging a more focused analysis.

Concordance of whole slide imaging and conventional light microscopy for assessment of pathologic response following neoadjuvant therapy for lung cancer.

Pathologic response is an endpoint in many ongoing clinical trials for neoadjuvant regimens, including immune checkpoint blockade and chemotherapy. Whole slide scanning of glass slides generates high resolution digital images and allows for remote review and potential measurement with image analysis tools, but concordance of pathologic response assessment on digital scans compared to glass slides has yet to be evaluated. Such a validation goes beyond previous concordance studies which focused on establishing surgical pathology diagnoses, as it requires quantitative assessment of tumor, necrosis, and regression. Further, as pathologic response assessment is being used as an endpoint, such concordance studies have regulatory implications. The purpose of this study was two fold: firstly, to determine the concordance between pathologic response assessed on glass slides and on digital scans; and secondly, to determine if pathologists benefited from using measurement tools when determining pathologic response. To that end, H&E-stained glass slides from 64 non-small cell lung carcinoma specimens were visually assessed for percent residual viable tumor (%RVT). The sensitivity and specificity for digital vs. glass reads of complete pathologic response (pCR, 0% RVT) and major pathologic response (MPR, ≤10% RVT) were all >95%. When %RVT was considered as a continuous variable, intraclass correlation coefficient of digital vs. glass reads was 0.94. The visual assessments of pathologic response were supported by pathologist annotations of residual tumor and tumor bed areas. In a separate subset of H&E-stained glass slides, several measurement approaches to quantifying %RVT were performed. Pathologist estimates strongly reflected measured %RVT. This study demonstrates the high level of concordance between glass slides evaluated using light microscopy and digital whole slide images for pathologic response assessments. Pathologists did not require measurement tools to generate robust %RVT values from slide annotations. These findings have broad implications for improving clinical workflows and multisite clinical trials.

Computational methods and biomarker discovery strategies for spatial proteomics: a review in immuno-oncology.

Advancements in imaging technologies have revolutionized our ability to deeply profile pathological tissue architectures, generating large volumes of imaging data with unparalleled spatial resolution. This type of data collection, namely, spatial proteomics, offers invaluable insights into various human diseases. Simultaneously, computational algorithms have evolved to manage the increasing dimensionality of spatial proteomics inherent in this progress. Numerous imaging-based computational frameworks, such as computational pathology, have been proposed for research and clinical applications. However, the development of these fields demands diverse domain expertise, creating barriers to their integration and further application. This review seeks to bridge this divide by presenting a comprehensive guideline. We consolidate prevailing computational methods and outline a roadmap from image processing to data-driven, statistics-informed biomarker discovery. Additionally, we explore future perspectives as the field moves toward interfacing with other quantitative domains, holding significant promise for precision care in immuno-oncology.

Characterization of Non-Monotonic Relationships between Tumor Mutational Burden and Clinical Outcomes.

Potential clinical biomarkers are often assessed with Cox regressions or their ability to differentiate two groups of patients based on a single cutoff. However, both of these approaches assume a monotonic relationship between the potential biomarker and survival. Tumor mutational burden (TMB) is currently being studied as a predictive biomarker for immunotherapy, and a single cutoff is often used to divide patients. In this study, we introduce a two-cutoff approach that allows splitting of patients when a non-monotonic relationship is present and explore the use of neural networks to model more complex relationships of TMB to outcome data. Using real-world data, we find that while in most cases the true relationship between TMB and survival appears monotonic, that is not always the case and researchers should be made aware of this possibility. SIGNIFICANCE: When a non-monotonic relationship to survival is present it is not possible to divide patients by a single value of a predictor. Neural networks allow for complex transformations and can be used to correctly split patients when a non-monotonic relationship is present.

Analysis of 3D pathology samples using weakly supervised AI.

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

Multiple-instance learning of somatic mutations for the classification of tumour type and the prediction of microsatellite status.

Large-scale genomic data are well suited to analysis by deep learning algorithms. However, for many genomic datasets, labels are at the level of the sample rather than for individual genomic measures. Machine learning models leveraging these datasets generate predictions by using statically encoded measures that are then aggregated at the sample level. Here we show that a single weakly supervised end-to-end multiple-instance-learning model with multi-headed attention can be trained to encode and aggregate the local sequence context or genomic position of somatic mutations, hence allowing for the modelling of the importance of individual measures for sample-level classification and thus providing enhanced explainability. The model solves synthetic tasks that conventional models fail at, and achieves best-in-class performance for the classification of tumour type and for predicting microsatellite status. By improving the performance of tasks that require aggregate information from genomic datasets, multiple-instance deep learning may generate biological insight.

Precision Oncology Core Data Model to Support Clinical Genomics Decision Making.

PURPOSE: Precision oncology mandates developing standardized common data models (CDMs) to facilitate analyses and enable clinical decision making. Expert-opinion-based precision oncology initiatives are epitomized in Molecular Tumor Boards (MTBs), which process large volumes of clinical-genomic data to match genotypes with molecularly guided therapies. METHODS: We used the Johns Hopkins University MTB as a use case and developed a precision oncology core data model (Precision-DM) to capture key clinical-genomic data elements. We leveraged existing CDMs, building upon the Minimal Common Oncology Data Elements model (mCODE). Our model was defined as a set of profiles with multiple data elements, focusing on next-generation sequencing and variant annotations. Most elements were mapped to terminologies or code sets and the Fast Healthcare Interoperability Resources (FHIR). We subsequently compared our Precision-DM with existing CDMs, including the National Cancer Institute's Genomic Data Commons (NCI GDC), mCODE, OSIRIS, the clinical Genome Data Model (cGDM), and the genomic CDM (gCDM). RESULTS: Precision-DM contained 16 profiles and 355 data elements. 39% of the elements derived values from selected terminologies or code sets, and 61% were mapped to FHIR. Although we used most elements contained in mCODE, we significantly expanded the profiles to include genomic annotations, resulting in a partial overlap of 50.7% between our core model and mCODE. Limited overlap was noted between Precision-DM and OSIRIS (33.2%), NCI GDC (21.4%), cGDM (9.3%), and gCDM (7.9%). Precision-DM covered most of the mCODE elements (87.7%), with less coverage for OSIRIS (35.8%), NCI GDC (11%), cGDM (26%) and gCDM (33.3%). CONCLUSION: Precision-DM supports clinical-genomic data standardization to support the MTB use case and may allow for harmonized data pulls across health care systems, academic institutions, and community medical centers.

Probabilistic Mixture Models Improve Calibration of Panel-derived Tumor Mutational Burden in the Context of both Tumor-normal and Tumor-only Sequencing.

Background: Tumor mutational burden (TMB) has been investigated as a biomarker for immune checkpoint blockade (ICB) therapy. Increasingly, TMB is being estimated with gene panel-based assays (as opposed to full exome sequencing) and different gene panels cover overlapping but distinct genomic coordinates, making comparisons across panels difficult. Previous studies have suggested that standardization and calibration to exome-derived TMB be done for each panel to ensure comparability. With TMB cutoffs being developed from panel-based assays, there is a need to understand how to properly estimate exomic TMB values from different panel-based assays. Design: Our approach to calibration of panel-derived TMB to exomic TMB proposes the use of probabilistic mixture models that allow for nonlinear relationships along with heteroscedastic error. We examined various inputs including nonsynonymous, synonymous, and hotspot counts along with genetic ancestry. Using The Cancer Genome Atlas cohort, we generated a tumor-only version of the panel-restricted data by reintroducing private germline variants. Results: We were able to model more accurately the distribution of both tumor-normal and tumor-only data using the proposed probabilistic mixture models as compared with linear regression. Applying a model trained on tumor-normal data to tumor-only input results in biased TMB predictions. Including synonymous mutations resulted in better regression metrics across both data types, but ultimately a model able to dynamically weight the various input mutation types exhibited optimal performance. Including genetic ancestry improved model performance only in the context of tumor-only data, wherein private germline variants are observed. Significance: A probabilistic mixture model better models the nonlinearity and heteroscedasticity of the data as compared with linear regression. Tumor-only panel data are needed to properly calibrate tumor-only panels to exomic TMB. Leveraging the uncertainty of point estimates from these models better informs cohort stratification in terms of TMB.

Deep learning reveals predictive sequence concepts within immune repertoires to immunotherapy.

T cell receptor (TCR) sequencing has been used to characterize the immune response to cancer. However, most analyses have been restricted to quantitative measures such as clonality that do not leverage the complementarity-determining region 3 (CDR3) sequence. We use DeepTCR, a framework of deep learning algorithms, to reveal sequence concepts that are predictive of response to immunotherapy. We demonstrate that DeepTCR can predict response and use the model to infer the antigenic specificities of the predictive signature and their unique dynamics during therapy. The predictive signature of nonresponse is associated with high frequencies of TCRs predicted to recognize tumor-specific antigens, and these tumor-specific TCRs undergo a higher degree of dynamic changes on therapy in nonresponders versus responders. These results are consistent with a biological model where the hallmark of nonresponders is an accumulation of tumor-specific T cells that undergo turnover on therapy, possibly because of the dysfunctional state of these T cells in nonresponders.

Genomic Landscapes and Hallmarks of Mutant RAS in Human Cancers.

The RAS family of small GTPases represents the most commonly activated oncogenes in human cancers. To better understand the prevalence of somatic RAS mutations and the compendium of genes that are coaltered in RAS-mutant tumors, we analyzed targeted next-generation sequencing data of 607,863 mutations from 66,372 tumors in 51 cancer types in the AACR Project GENIE Registry. Bayesian hierarchical models were implemented to estimate the cancer-specific prevalence of RAS and non-RAS somatic mutations, to evaluate co-occurrence and mutual exclusivity, and to model the effects of tumor mutation burden and mutational signatures on comutation patterns. These analyses revealed differential RAS prevalence and comutations with non-RAS genes in a cancer lineage-dependent and context-dependent manner, with differences across age, sex, and ethnic groups. Allele-specific RAS co-mutational patterns included an enrichment in NTRK3 and chromatin-regulating gene mutations in KRAS G12C-mutant non-small cell lung cancer. Integrated multiomic analyses of 10,217 tumors from The Cancer Genome Atlas (TCGA) revealed distinct genotype-driven gene expression programs pointing to differential recruitment of cancer hallmarks as well as phenotypic differences and immune surveillance states in the tumor microenvironment of RAS-mutant tumors. The distinct genomic tracks discovered in RAS-mutant tumors reflected differential clinical outcomes in TCGA cohort and in an independent cohort of patients with KRAS G12C-mutant non-small cell lung cancer that received immunotherapy-containing regimens. The RAS genetic architecture points to cancer lineage-specific therapeutic vulnerabilities that can be leveraged for rationally combining RAS-mutant allele-directed therapies with targeted therapies and immunotherapy. SIGNIFICANCE: The complex genomic landscape of RAS-mutant tumors is reflective of selection processes in a cancer lineage-specific and context-dependent manner, highlighting differential therapeutic vulnerabilities that can be clinically translated.

Predictive models of response to neoadjuvant chemotherapy in muscle-invasive bladder cancer using nuclear morphology and tissue architecture.

Characterizing likelihood of response to neoadjuvant chemotherapy (NAC) in muscle-invasive bladder cancer (MIBC) is an important yet unmet challenge. In this study, a machine-learning framework is developed using imaging of biopsy pathology specimens to generate models of likelihood of NAC response. Developed using cross-validation (evaluable N = 66) and an independent validation cohort (evaluable N = 56), our models achieve promising results (65%-73% accuracy). Interestingly, one model-using features derived from hematoxylin and eosin (H&E)-stained tissues in conjunction with clinico-demographic features-is able to stratify the cohort into likely responders in cross-validation and the validation cohort (response rate of 65% for predicted responder compared with the 41% baseline response rate in the validation cohort). The results suggest that computational approaches applied to routine pathology specimens of MIBC can capture differences between responders and non-responders to NAC and should therefore be considered in the future design of precision oncology for MIBC.

Spatial UMAP and Image Cytometry for Topographic Immuno-oncology Biomarker Discovery.

Multiplex immunofluorescence (mIF) can detail spatial relationships and complex cell phenotypes in the tumor microenvironment (TME). However, the analysis and visualization of mIF data can be complex and time-consuming. Here, we used tumor specimens from 93 patients with metastatic melanoma to develop and validate a mIF data analysis pipeline using established flow cytometry workflows (image cytometry). Unlike flow cytometry, spatial information from the TME was conserved at single-cell resolution. A spatial uniform manifold approximation and projection (UMAP) was constructed using the image cytometry output. Spatial UMAP subtraction analysis (survivors vs. nonsurvivors at 5 years) was used to identify topographic and coexpression signatures with positive or negative prognostic impact. Cell densities and proportions identified by image cytometry showed strong correlations when compared with those obtained using gold-standard, digital pathology software (R2 > 0.8). The associated spatial UMAP highlighted "immune neighborhoods" and associated topographic immunoactive protein expression patterns. We found that PD-L1 and PD-1 expression intensity was spatially encoded-the highest PD-L1 expression intensity was observed on CD163+ cells in neighborhoods with high CD8+ cell density, and the highest PD-1 expression intensity was observed on CD8+ cells in neighborhoods with dense arrangements of tumor cells. Spatial UMAP subtraction analysis revealed numerous spatial clusters associated with clinical outcome. The variables represented in the key clusters from the unsupervised UMAP analysis were validated using established, supervised approaches. In conclusion, image cytometry and the spatial UMAPs presented herein are powerful tools for the visualization and interpretation of single-cell, spatially resolved mIF data and associated topographic biomarker development.

Deep learning identifies antigenic determinants of severe SARS-CoV-2 infection within T-cell repertoires.

SARS-CoV-2 infection is characterized by a highly variable clinical course with patients experiencing asymptomatic infection all the way to requiring critical care support. This variation in clinical course has led physicians and scientists to study factors that may predispose certain individuals to more severe clinical presentations in hopes of either identifying these individuals early in their illness or improving their medical management. We sought to understand immunogenomic differences that may result in varied clinical outcomes through analysis of T-cell receptor sequencing (TCR-Seq) data in the open access ImmuneCODE database. We identified two cohorts within the database that had clinical outcomes data reflecting severity of illness and utilized DeepTCR, a multiple-instance deep learning repertoire classifier, to predict patients with severe SARS-CoV-2 infection from their repertoire sequencing. We demonstrate that patients with severe infection have repertoires with higher T-cell responses associated with SARS-CoV-2 epitopes and identify the epitopes that result in these responses. Our results provide evidence that the highly variable clinical course seen in SARS-CoV-2 infection is associated to certain antigen-specific responses.

Deep learning for diagnosis of acute promyelocytic leukemia via recognition of genomically imprinted morphologic features.

Acute promyelocytic leukemia (APL) is a subtype of acute myeloid leukemia (AML), classified by a translocation between chromosomes 15 and 17 [t(15;17)], that is considered a true oncologic emergency though appropriate therapy is considered curative. Therapy is often initiated on clinical suspicion, informed by both clinical presentation as well as direct visualization of the peripheral smear. We hypothesized that genomic imprinting of morphologic features learned by deep learning pattern recognition would have greater discriminatory power and consistency compared to humans, thereby facilitating identification of t(15;17) positive APL. By applying both cell-level and patient-level classification linked to t(15;17) PML/RARA ground-truth, we demonstrate that deep learning is capable of distinguishing APL in both discovery and prospective independent cohort of patients. Furthermore, we extract learned information from the trained network to identify previously undescribed morphological features of APL. The deep learning method we describe herein potentially allows a rapid, explainable, and accurate physician-aid for diagnosing APL at the time of presentation in any resource-poor or -rich medical setting given the universally available peripheral smear.

Identification of BXDC2 as a Key Downstream Effector of the Androgen Receptor in Modulating Cisplatin Sensitivity in Bladder Cancer.

Underlying mechanisms for resistance to cisplatin-based chemotherapy in bladder cancer patients are largely unknown, although androgen receptor (AR) activity, as well as extracellular signal-regulated kinase (ERK) signaling, has been indicated to correlate with chemosensitivity. We also previously showed ERK activation by androgen treatment in AR-positive bladder cancer cells. Because our DNA microarray analysis in control vs. AR-knockdown bladder cancer lines identified BXDC2 as a potential downstream target of AR, we herein assessed its functional role in cisplatin sensitivity, using bladder cancer lines and surgical specimens. BXDC2 protein expression was considerably downregulated in AR-positive or cisplatin-resistant cells. BXDC2-knockdown sublines were significantly more resistant to cisplatin, compared with respective controls. Without cisplatin treatment, BXDC2-knockdown resulted in significant increases/decreases in cell proliferation/apoptosis, respectively. An ERK activator was also found to reduce BXDC2 expression. Immunohistochemistry showed downregulation of BXDC2 expression in tumor (vs. non-neoplastic urothelium), higher grade/stage tumor (vs. lower grade/stage), and AR-positive tumor (vs. AR-negative). Patients with BXDC2-positive/AR-negative muscle-invasive bladder cancer had a significantly lower risk of disease-specific mortality, compared to those with a BXDC2-negative/AR-positive tumor. Additionally, in those undergoing cisplatin-based chemotherapy, BXDC2 positivity alone (p = 0.083) or together with AR negativity (p = 0.047) was associated with favorable response. We identified BXDC2 as a key molecule in enhancing cisplatin sensitivity. AR-ERK activation may thus be associated with chemoresistance via downregulating BXDC2 expression in bladder cancer.

DeepTCR is a deep learning framework for revealing sequence concepts within T-cell repertoires.

Deep learning algorithms have been utilized to achieve enhanced performance in pattern-recognition tasks. The ability to learn complex patterns in data has tremendous implications in immunogenomics. T-cell receptor (TCR) sequencing assesses the diversity of the adaptive immune system and allows for modeling its sequence determinants of antigenicity. We present DeepTCR, a suite of unsupervised and supervised deep learning methods able to model highly complex TCR sequencing data by learning a joint representation of a TCR by its CDR3 sequences and V/D/J gene usage. We demonstrate the utility of deep learning to provide an improved 'featurization' of the TCR across multiple human and murine datasets, including improved classification of antigen-specific TCRs and extraction of antigen-specific TCRs from noisy single-cell RNA-Seq and T-cell culture-based assays. Our results highlight the flexibility and capacity for deep neural networks to extract meaningful information from complex immunogenomic data for both descriptive and predictive purposes.

College of American Pathologists Cancer Protocols: From Optimizing Cancer Patient Care to Facilitating Interoperable Reporting and Downstream Data Use.

The College of American Pathologists Cancer Protocols have offered guidance to pathologists for standard cancer pathology reporting for more than 35 years. The adoption of computer readable versions of these protocols by electronic health record and laboratory information system (LIS) vendors has provided a mechanism for pathologists to report within their LIS workflow, in addition to enabling standardized structured data capture and reporting to downstream consumers of these data such as the cancer surveillance community. This paper reviews the history of the Cancer Protocols and electronic Cancer Checklists, outlines the current use of these critically important cancer case reporting tools, and examines future directions, including plans to help improve the integration of the Cancer Protocols into clinical, public health, research, and other workflows.

Integrative Tumor and Immune Cell Multi-omic Analyses Predict Response to Immune Checkpoint Blockade in Melanoma.

In this study, we incorporate analyses of genome-wide sequence and structural alterations with pre- and on-therapy transcriptomic and T cell repertoire features in immunotherapy-naive melanoma patients treated with immune checkpoint blockade. Although tumor mutation burden is associated with improved treatment response, the mutation frequency in expressed genes is superior in predicting outcome. Increased T cell density in baseline tumors and dynamic changes in regression or expansion of the T cell repertoire during therapy distinguish responders from non-responders. Transcriptome analyses reveal an increased abundance of B cell subsets in tumors from responders and patterns of molecular response related to expressed mutation elimination or retention that reflect clinical outcome. High-dimensional genomic, transcriptomic, and immune repertoire data were integrated into a multi-modal predictor of response. These findings identify genomic and transcriptomic characteristics of tumors and immune cells that predict response to immune checkpoint blockade and highlight the importance of pre-existing T and B cell immunity in therapeutic outcomes.

Multimodal genomic features predict outcome of immune checkpoint blockade in non-small-cell lung cancer.

Despite progress in immunotherapy, identifying patients that respond has remained a challenge. Through analysis of whole-exome and targeted sequence data from 5,449 tumors, we found a significant correlation between tumor mutation burden (TMB) and tumor purity, suggesting that low tumor purity tumors are likely to have inaccurate TMB estimates. We developed a new method to estimate a corrected TMB (cTMB) that was adjusted for tumor purity and more accurately predicted outcome to immune checkpoint blockade (ICB). To identify improved predictive markers together with cTMB, we performed whole-exome sequencing for 104 lung tumors treated with ICB. Through comprehensive analyses of sequence and structural alterations, we discovered a significant enrichment in activating mutations in receptor tyrosine kinase (RTK) genes in nonresponding tumors in three immunotherapy treated cohorts. An integrated multivariable model incorporating cTMB, RTK mutations, smoking-related mutational signature and human leukocyte antigen status provided an improved predictor of response to immunotherapy that was independently validated.