Paired evaluation of machine-learning models characterizes effects of confounders and outliers.

The true accuracy of a machine-learning model is a population-level statistic that cannot be observed directly. In practice, predictor performance is estimated against one or more test datasets, and the accuracy of this estimate strongly depends on how well the test sets represent all possible unseen datasets. Here we describe paired evaluation as a simple, robust approach for evaluating performance of machine-learning models in small-sample biological and clinical studies. We use the method to evaluate predictors of drug response in breast cancer cell lines and of disease severity in patients with Alzheimer's disease, demonstrating that the choice of test data can cause estimates of performance to vary by as much as 20%. We show that paired evaluation makes it possible to identify outliers, improve the accuracy of performance estimates in the presence of known confounders, and assign statistical significance when comparing machine-learning models.

Addressing persistent challenges in digital image analysis of cancerous tissues.

The National Cancer Institute (NCI) supports many research programs and consortia, many of which use imaging as a major modality for characterizing cancerous tissue. A trans-consortia Image Analysis Working Group (IAWG) was established in 2019 with a mission to disseminate imaging-related work and foster collaborations. In 2022, the IAWG held a virtual hackathon focused on addressing challenges of analyzing high dimensional datasets from fixed cancerous tissues. Standard image processing techniques have automated feature extraction, but the next generation of imaging data requires more advanced methods to fully utilize the available information. In this perspective, we discuss current limitations of the automated analysis of multiplexed tissue images, the first steps toward deeper understanding of these limitations, what possible solutions have been developed, any new or refined approaches that were developed during the Image Analysis Hackathon 2022, and where further effort is required. The outstanding problems addressed in the hackathon fell into three main themes: 1) challenges to cell type classification and assessment, 2) translation and visual representation of spatial aspects of high dimensional data, and 3) scaling digital image analyses to large (multi-TB) datasets. We describe the rationale for each specific challenge and the progress made toward addressing it during the hackathon. We also suggest areas that would benefit from more focus and offer insight into broader challenges that the community will need to address as new technologies are developed and integrated into the broad range of image-based modalities and analytical resources already in use within the cancer research community.

Early on-treatment transcriptional profiling as a tool for improving pathological response prediction in HER2-positive inflammatory breast cancer.

Background: Inflammatory breast cancer (IBC) is a rare and understudied disease, with 40% of cases presenting with human epidermal growth factor receptor 2 (HER2)-positive subtype. The goals of this study were to (i) assess the pathologic complete response (pCR) rate of short-term neoadjuvant dual-HER2-blockade and paclitaxel, (ii) contrast baseline and on-treatment transcriptional profiles of IBC tumor biopsies associated with pCR, and (iii) identify biological pathways that may explain the effect of neoadjuvant therapy on tumor response. Patients and Methods: A single-arm phase II trial of neoadjuvant trastuzumab (H), pertuzumab (P), and paclitaxel for 16 weeks was completed among patients with newly diagnosed HER2-positive IBC. Fresh-frozen tumor biopsies were obtained pretreatment (D1) and 8 days later (D8), following a single dose of HP, prior to adding paclitaxel. We performed RNA-sequencing on D1 and D8 tumor biopsies, identified genes associated with pCR using differential gene expression analysis, identified pathways associated with pCR using gene set enrichment and gene expression deconvolution methods, and compared the pCR predictive value of principal components derived from gene expression profiles by calculating and area under the curve for D1 and D8 subsets. Results: Twenty-three participants were enrolled, of whom 21 completed surgery following neoadjuvant therapy. Paired longitudinal fresh-frozen tumor samples (D1 and D8) were obtained from all patients. Among the 21 patients who underwent surgery, the pCR and the 4-year disease-free survival were 48% (90% CI 0.29-0.67) and 90% (95% CI 66-97%), respectively. The transcriptional profile of D8 biopsies was found to be more predictive of pCR (AUC = 0.91, 95% CI: 0.7993-1) than the D1 biopsies (AUC = 0.79, 95% CI: 0.5905-0.9822). Conclusions: In patients with HER2-positive IBC treated with neoadjuvant HP and paclitaxel for 16 weeks, gene expression patterns of tumor biopsies measured 1 week after treatment initiation not only offered different biological information but importantly served as a better predictor of pCR than baseline transcriptional analysis. Trial Registration: ClinicalTrials.gov identifier: NCT01796197 (https://clinicaltrials.gov/ct2/show/NCT01796197); registered on February 21, 2013.

A community-based approach to image analysis of cells, tissues and tumors.

Emerging multiplexed imaging platforms provide an unprecedented view of an increasing number of molecular markers at subcellular resolution and the dynamic evolution of tumor cellular composition. As such, they are capable of elucidating cell-to-cell interactions within the tumor microenvironment that impact clinical outcome and therapeutic response. However, the rapid development of these platforms has far outpaced the computational methods for processing and analyzing the data they generate. While being technologically disparate, all imaging assays share many computational requirements for post-collection data processing. As such, our Image Analysis Working Group (IAWG), composed of researchers in the Cancer Systems Biology Consortium (CSBC) and the Physical Sciences - Oncology Network (PS-ON), convened a workshop on "Computational Challenges Shared by Diverse Imaging Platforms" to characterize these common issues and a follow-up hackathon to implement solutions for a selected subset of them. Here, we delineate these areas that reflect major axes of research within the field, including image registration, segmentation of cells and subcellular structures, and identification of cell types from their morphology. We further describe the logistical organization of these events, believing our lessons learned can aid others in uniting the imaging community around self-identified topics of mutual interest, in designing and implementing operational procedures to address those topics and in mitigating issues inherent in image analysis (e.g., sharing exemplar images of large datasets and disseminating baseline solutions to hackathon challenges through open-source code repositories).

MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging.

Highly multiplexed tissue imaging makes detailed molecular analysis of single cells possible in a preserved spatial context. However, reproducible analysis of large multichannel images poses a substantial computational challenge. Here, we describe a modular and open-source computational pipeline, MCMICRO, for performing the sequential steps needed to transform whole-slide images into single-cell data. We demonstrate the use of MCMICRO on tissue and tumor images acquired using multiple imaging platforms, thereby providing a solid foundation for the continued development of tissue imaging software.

A serum-based DNA methylation assay provides accurate detection of glioma.

BACKGROUND: The detection of somatic mutations in cell-free DNA (cfDNA) from liquid biopsy has emerged as a noninvasive tool to monitor the follow-up of cancer patients. However, the significance of cfDNA clinical utility remains uncertain in patients with brain tumors, primarily because of the limited sensitivity cfDNA has to detect real tumor-specific somatic mutations. This unresolved challenge has prevented accurate follow-up of glioma patients with noninvasive approaches. METHODS: Genome-wide DNA methylation profiling of tumor tissue and serum cfDNA of glioma patients. RESULTS: Here, we developed a noninvasive approach to profile the DNA methylation status in the serum of patients with gliomas and identified a cfDNA-derived methylation signature that is associated with the presence of gliomas and related immune features. By testing the signature in an independent discovery and validation cohorts, we developed and verified a score metric (the "glioma-epigenetic liquid biopsy score" or GeLB) that optimally distinguished patients with or without glioma (sensitivity: 100%, specificity: 97.78%). Furthermore, we found that changes in GeLB score reflected clinicopathological changes during surveillance (eg, progression, pseudoprogression, and response to standard or experimental treatment). CONCLUSIONS: Our results suggest that the GeLB score can be used as a complementary approach to diagnose and follow up patients with glioma.

Machine learning identifies candidates for drug repurposing in Alzheimer's disease.

Clinical trials of novel therapeutics for Alzheimer's Disease (AD) have consumed a large amount of time and resources with largely negative results. Repurposing drugs already approved by the Food and Drug Administration (FDA) for another indication is a more rapid and less expensive option. We present DRIAD (Drug Repurposing In AD), a machine learning framework that quantifies potential associations between the pathology of AD severity (the Braak stage) and molecular mechanisms as encoded in lists of gene names. DRIAD is applied to lists of genes arising from perturbations in differentiated human neural cell cultures by 80 FDA-approved and clinically tested drugs, producing a ranked list of possible repurposing candidates. Top-scoring drugs are inspected for common trends among their targets. We propose that the DRIAD method can be used to nominate drugs that, after additional validation and identification of relevant pharmacodynamic biomarker(s), could be readily evaluated in a clinical trial.

Facetto: Combining Unsupervised and Supervised Learning for Hierarchical Phenotype Analysis in Multi-Channel Image Data.

Facetto is a scalable visual analytics application that is used to discover single-cell phenotypes in high-dimensional multi-channel microscopy images of human tumors and tissues. Such images represent the cutting edge of digital histology and promise to revolutionize how diseases such as cancer are studied, diagnosed, and treated. Highly multiplexed tissue images are complex, comprising 109 or more pixels, 60-plus channels, and millions of individual cells. This makes manual analysis challenging and error-prone. Existing automated approaches are also inadequate, in large part, because they are unable to effectively exploit the deep knowledge of human tissue biology available to anatomic pathologists. To overcome these challenges, Facetto enables a semi-automated analysis of cell types and states. It integrates unsupervised and supervised learning into the image and feature exploration process and offers tools for analytical provenance. Experts can cluster the data to discover new types of cancer and immune cells and use clustering results to train a convolutional neural network that classifies new cells accordingly. Likewise, the output of classifiers can be clustered to discover aggregate patterns and phenotype subsets. We also introduce a new hierarchical approach to keep track of analysis steps and data subsets created by users; this assists in the identification of cell types. Users can build phenotype trees and interact with the resulting hierarchical structures of both high-dimensional feature and image spaces. We report on use-cases in which domain scientists explore various large-scale fluorescence imaging datasets. We demonstrate how Facetto assists users in steering the clustering and classification process, inspecting analysis results, and gaining new scientific insights into cancer biology.

Highly multiplexed immunofluorescence images and single-cell data of immune markers in tonsil and lung cancer.

In this data descriptor, we document a dataset of multiplexed immunofluorescence images and derived single-cell measurements of immune lineage and other markers in formaldehyde-fixed and paraffin-embedded (FFPE) human tonsil and lung cancer tissue. We used tissue cyclic immunofluorescence (t-CyCIF) to generate fluorescence images which we artifact corrected using the BaSiC tool, stitched and registered using the ASHLAR algorithm, and segmented using ilastik software and MATLAB. We extracted single-cell features from these images using HistoCAT software. The resulting dataset can be visualized using image browsers and analyzed using high-dimensional, single-cell methods. This dataset is a valuable resource for biological discovery of the immune system in normal and diseased states as well as for the development of multiplexed image analysis and viewing tools.

Clinical and Genomic Characterization of Treatment-Emergent Small-Cell Neuroendocrine Prostate Cancer: A Multi-institutional Prospective Study.

Purpose The prevalence and features of treatment-emergent small-cell neuroendocrine prostate cancer (t-SCNC) are not well characterized in the era of modern androgen receptor (AR)-targeting therapy. We sought to characterize the clinical and genomic features of t-SCNC in a multi-institutional prospective study. Methods Patients with progressive, metastatic castration-resistant prostate cancer (mCRPC) underwent metastatic tumor biopsy and were followed for survival. Metastatic biopsy specimens underwent independent, blinded pathology review along with RNA/DNA sequencing. Results A total of 202 consecutive patients were enrolled. One hundred forty-eight (73%) had prior disease progression on abiraterone and/or enzalutamide. The biopsy evaluable rate was 79%. The overall incidence of t-SCNC detection was 17%. AR amplification and protein expression were present in 67% and 75%, respectively, of t-SCNC biopsy specimens. t-SCNC was detected at similar proportions in bone, node, and visceral organ biopsy specimens. Genomic alterations in the DNA repair pathway were nearly mutually exclusive with t-SCNC differentiation ( P = .035). Detection of t-SCNC was associated with shortened overall survival among patients with prior AR-targeting therapy for mCRPC (hazard ratio, 2.02; 95% CI, 1.07 to 3.82). Unsupervised hierarchical clustering of the transcriptome identified a small-cell-like cluster that further enriched for adverse survival outcomes (hazard ratio, 3.00; 95% CI, 1.25 to 7.19). A t-SCNC transcriptional signature was developed and validated in multiple external data sets with > 90% accuracy. Multiple transcriptional regulators of t-SCNC were identified, including the pancreatic neuroendocrine marker PDX1. Conclusion t-SCNC is present in nearly one fifth of patients with mCRPC and is associated with shortened survival. The near-mutual exclusivity with DNA repair alterations suggests t-SCNC may be a distinct subset of mCRPC. Transcriptional profiling facilitates the identification of t-SCNC and novel therapeutic targets.

Machine Learning Identifies Stemness Features Associated with Oncogenic Dedifferentiation.

Cancer progression involves the gradual loss of a differentiated phenotype and acquisition of progenitor and stem-cell-like features. Here, we provide novel stemness indices for assessing the degree of oncogenic dedifferentiation. We used an innovative one-class logistic regression (OCLR) machine-learning algorithm to extract transcriptomic and epigenetic feature sets derived from non-transformed pluripotent stem cells and their differentiated progeny. Using OCLR, we were able to identify previously undiscovered biological mechanisms associated with the dedifferentiated oncogenic state. Analyses of the tumor microenvironment revealed unanticipated correlation of cancer stemness with immune checkpoint expression and infiltrating immune cells. We found that the dedifferentiated oncogenic phenotype was generally most prominent in metastatic tumors. Application of our stemness indices to single-cell data revealed patterns of intra-tumor molecular heterogeneity. Finally, the indices allowed for the identification of novel targets and possible targeted therapies aimed at tumor differentiation.

A Distinct DNA Methylation Shift in a Subset of Glioma CpG Island Methylator Phenotypes during Tumor Recurrence.

Glioma diagnosis is based on histomorphology and grading; however, such classification does not have predictive clinical outcome after glioblastomas have developed. To date, no bona fide biomarkers that significantly translate into a survival benefit to glioblastoma patients have been identified. We previously reported that the IDH mutant G-CIMP-high subtype would be a predecessor to the G-CIMP-low subtype. Here, we performed a comprehensive DNA methylation longitudinal analysis of diffuse gliomas from 77 patients (200 tumors) to enlighten the epigenome-based malignant transformation of initially lower-grade gliomas. Intra-subtype heterogeneity among G-CIMP-high primary tumors allowed us to identify predictive biomarkers for assessing the risk of malignant recurrence at early stages of disease. G-CIMP-low recurrence appeared in 9.5% of all gliomas, and these resembled IDH-wild-type primary glioblastoma. G-CIMP-low recurrence can be characterized by distinct epigenetic changes at candidate functional tissue enhancers with AP-1/SOX binding elements, mesenchymal stem cell-like epigenomic phenotype, and genomic instability. Molecular abnormalities of longitudinal G-CIMP offer possibilities to defy glioblastoma progression.

Prophetic Granger Causality to infer gene regulatory networks.

We introduce a novel method called Prophetic Granger Causality (PGC) for inferring gene regulatory networks (GRNs) from protein-level time series data. The method uses an L1-penalized regression adaptation of Granger Causality to model protein levels as a function of time, stimuli, and other perturbations. When combined with a data-independent network prior, the framework outperformed all other methods submitted to the HPN-DREAM 8 breast cancer network inference challenge. Our investigations reveal that PGC provides complementary information to other approaches, raising the performance of ensemble learners, while on its own achieves moderate performance. Thus, PGC serves as a valuable new tool in the bioinformatics toolkit for analyzing temporal datasets. We investigate the general and cell-specific interactions predicted by our method and find several novel interactions, demonstrating the utility of the approach in charting new tumor wiring.

A Community Challenge for Inferring Genetic Predictors of Gene Essentialities through Analysis of a Functional Screen of Cancer Cell Lines.

We report the results of a DREAM challenge designed to predict relative genetic essentialities based on a novel dataset testing 98,000 shRNAs against 149 molecularly characterized cancer cell lines. We analyzed the results of over 3,000 submissions over a period of 4 months. We found that algorithms combining essentiality data across multiple genes demonstrated increased accuracy; gene expression was the most informative molecular data type; the identity of the gene being predicted was far more important than the modeling strategy; well-predicted genes and selected molecular features showed enrichment in functional categories; and frequently selected expression features correlated with survival in primary tumors. This study establishes benchmarks for gene essentiality prediction, presents a community resource for future comparison with this benchmark, and provides insights into factors influencing the ability to predict gene essentiality from functional genetic screens. This study also demonstrates the value of releasing pre-publication data publicly to engage the community in an open research collaboration.

N-Myc Drives Neuroendocrine Prostate Cancer Initiated from Human Prostate Epithelial Cells.

MYCN amplification and overexpression are common in neuroendocrine prostate cancer (NEPC). However, the impact of aberrant N-Myc expression in prostate tumorigenesis and the cellular origin of NEPC have not been established. We define N-Myc and activated AKT1 as oncogenic components sufficient to transform human prostate epithelial cells to prostate adenocarcinoma and NEPC with phenotypic and molecular features of aggressive, late-stage human disease. We directly show that prostate adenocarcinoma and NEPC can arise from a common epithelial clone. Further, N-Myc is required for tumor maintenance, and destabilization of N-Myc through Aurora A kinase inhibition reduces tumor burden. Our findings establish N-Myc as a driver of NEPC and a target for therapeutic intervention.

Pathway-Based Genomics Prediction using Generalized Elastic Net.

We present a novel regularization scheme called The Generalized Elastic Net (GELnet) that incorporates gene pathway information into feature selection. The proposed formulation is applicable to a wide variety of problems in which the interpretation of predictive features using known molecular interactions is desired. The method naturally steers solutions toward sets of mechanistically interlinked genes. Using experiments on synthetic data, we demonstrate that pathway-guided results maintain, and often improve, the accuracy of predictors even in cases where the full gene network is unknown. We apply the method to predict the drug response of breast cancer cell lines. GELnet is able to reveal genetic determinants of sensitivity and resistance for several compounds. In particular, for an EGFR/HER2 inhibitor, it finds a possible trans-differentiation resistance mechanism missed by the corresponding pathway agnostic approach.

Inferring causal molecular networks: empirical assessment through a community-based effort.

It remains unclear whether causal, rather than merely correlational, relationships in molecular networks can be inferred in complex biological settings. Here we describe the HPN-DREAM network inference challenge, which focused on learning causal influences in signaling networks. We used phosphoprotein data from cancer cell lines as well as in silico data from a nonlinear dynamical model. Using the phosphoprotein data, we scored more than 2,000 networks submitted by challenge participants. The networks spanned 32 biological contexts and were scored in terms of causal validity with respect to unseen interventional data. A number of approaches were effective, and incorporating known biology was generally advantageous. Additional sub-challenges considered time-course prediction and visualization. Our results suggest that learning causal relationships may be feasible in complex settings such as disease states. Furthermore, our scoring approach provides a practical way to empirically assess inferred molecular networks in a causal sense.

ONE-CLASS DETECTION OF CELL STATES IN TUMOR SUBTYPES.

The cellular composition of a tumor greatly influences the growth, spread, immune activity, drug response, and other aspects of the disease. Tumor cells are usually comprised of a heterogeneous mixture of subclones, each of which could contain their own distinct character. The presence of minor subclones poses a serious health risk for patients as any one of them could harbor a fitness advantage with respect to the current treatment regimen, fueling resistance. It is therefore vital to accurately assess the make-up of cell states within a tumor biopsy. Transcriptome-wide assays from RNA sequencing provide key data from which cell state signatures can be detected. However, the challenge is to find them within samples containing mixtures of cell types of unknown proportions. We propose a novel one-class method based on logistic regression and show that its performance is competitive to two established SVM-based methods for this detection task. We demonstrate that one-class models are able to identify specific cell types in heterogeneous cell populations better than their binary predictor counterparts. We derive one-class predictors for the major breast and bladder subtypes and reaffirm the connection between these two tissues. In addition, we use a one-class predictor to quantitatively associate an embryonic stem cell signature with an aggressive breast cancer subtype that reveals shared stemness pathways potentially important for treatment.

A basal stem cell signature identifies aggressive prostate cancer phenotypes.

Evidence from numerous cancers suggests that increased aggressiveness is accompanied by up-regulation of signaling pathways and acquisition of properties common to stem cells. It is unclear if different subtypes of late-stage cancer vary in stemness properties and whether or not these subtypes are transcriptionally similar to normal tissue stem cells. We report a gene signature specific for human prostate basal cells that is differentially enriched in various phenotypes of late-stage metastatic prostate cancer. We FACS-purified and transcriptionally profiled basal and luminal epithelial populations from the benign and cancerous regions of primary human prostates. High-throughput RNA sequencing showed the basal population to be defined by genes associated with stem cell signaling programs and invasiveness. Application of a 91-gene basal signature to gene expression datasets from patients with organ-confined or hormone-refractory metastatic prostate cancer revealed that metastatic small cell neuroendocrine carcinoma was molecularly more stem-like than either metastatic adenocarcinoma or organ-confined adenocarcinoma. Bioinformatic analysis of the basal cell and two human small cell gene signatures identified a set of E2F target genes common between prostate small cell neuroendocrine carcinoma and primary prostate basal cells. Taken together, our data suggest that aggressive prostate cancer shares a conserved transcriptional program with normal adult prostate basal stem cells.

Assessing the clinical utility of cancer genomic and proteomic data across tumor types.

Molecular profiling of tumors promises to advance the clinical management of cancer, but the benefits of integrating molecular data with traditional clinical variables have not been systematically studied. Here we retrospectively predict patient survival using diverse molecular data (somatic copy-number alteration, DNA methylation and mRNA, microRNA and protein expression) from 953 samples of four cancer types from The Cancer Genome Atlas project. We find that incorporating molecular data with clinical variables yields statistically significantly improved predictions (FDR < 0.05) for three cancers but those quantitative gains were limited (2.2-23.9%). Additional analyses revealed little predictive power across tumor types except for one case. In clinically relevant genes, we identified 10,281 somatic alterations across 12 cancer types in 2,928 of 3,277 patients (89.4%), many of which would not be revealed in single-tumor analyses. Our study provides a starting point and resources, including an open-access model evaluation platform, for building reliable prognostic and therapeutic strategies that incorporate molecular data.