Erratum: Paired evaluation of machine-learning models characterizes effects of confounders and outliers.

[This corrects the article DOI: 10.1016/j.patter.2023.100791.].

Paired evaluation of machine-learning models characterizes effects of confounders and outliers.

The true accuracy of a machine-learning model is a population-level statistic that cannot be observed directly. In practice, predictor performance is estimated against one or more test datasets, and the accuracy of this estimate strongly depends on how well the test sets represent all possible unseen datasets. Here we describe paired evaluation as a simple, robust approach for evaluating performance of machine-learning models in small-sample biological and clinical studies. We use the method to evaluate predictors of drug response in breast cancer cell lines and of disease severity in patients with Alzheimer's disease, demonstrating that the choice of test data can cause estimates of performance to vary by as much as 20%. We show that paired evaluation makes it possible to identify outliers, improve the accuracy of performance estimates in the presence of known confounders, and assign statistical significance when comparing machine-learning models.

Proteomic profiling across breast cancer cell lines and models.

We performed quantitative proteomics on 60 human-derived breast cancer cell line models to a depth of ~13,000 proteins. The resulting high-throughput datasets were assessed for quality and reproducibility. We used the datasets to identify and characterize the subtypes of breast cancer and showed that they conform to known transcriptional subtypes, revealing that molecular subtypes are preserved even in under-sampled protein feature sets. All datasets are freely available as public resources on the LINCS portal. We anticipate that these datasets, either in isolation or in combination with complimentary measurements such as genomics, transcriptomics and phosphoproteomics, can be mined for the purpose of predicting drug response, informing cell line specific context in models of signalling pathways, and identifying markers of sensitivity or resistance to therapeutics.

Targeting TBK1 to overcome resistance to cancer immunotherapy.

Despite the success of PD-1 blockade in melanoma and other cancers, effective treatment strategies to overcome resistance to cancer immunotherapy are lacking1,2. Here we identify the innate immune kinase TANK-binding kinase 1 (TBK1)3 as a candidate immune-evasion gene in a pooled genetic screen4. Using a suite of genetic and pharmacological tools across multiple experimental model systems, we confirm a role for TBK1 as an immune-evasion gene. Targeting TBK1 enhances responses to PD-1 blockade by decreasing the cytotoxicity threshold to effector cytokines (TNF and IFNγ). TBK1 inhibition in combination with PD-1 blockade also demonstrated efficacy using patient-derived tumour models, with concordant findings in matched patient-derived organotypic tumour spheroids and matched patient-derived organoids. Tumour cells lacking TBK1 are primed to undergo RIPK- and caspase-dependent cell death in response to TNF and IFNγ in a JAK-STAT-dependent manner. Taken together, our results demonstrate that targeting TBK1 is an effective strategy to overcome resistance to cancer immunotherapy.

Multiplexed and reproducible high content screening of live and fixed cells using Dye Drop.

High-throughput measurement of cells perturbed using libraries of small molecules, gene knockouts, or different microenvironmental factors is a key step in functional genomics and pre-clinical drug discovery. However, it remains difficult to perform accurate single-cell assays in 384-well plates, limiting many studies to well-average measurements (e.g., CellTiter-Glo®). Here we describe a public domain Dye Drop method that uses sequential density displacement and microscopy to perform multi-step assays on living cells. We use Dye Drop cell viability and DNA replication assays followed by immunofluorescence imaging to collect single-cell dose-response data for 67 investigational and clinical-grade small molecules in 58 breast cancer cell lines. By separating the cytostatic and cytotoxic effects of drugs computationally, we uncover unexpected relationships between the two. Dye Drop is rapid, reproducible, customizable, and compatible with manual or automated laboratory equipment. Dye Drop improves the tradeoff between data content and cost, enabling the collection of information-rich perturbagen-response datasets.

A multi-omic analysis of MCF10A cells provides a resource for integrative assessment of ligand-mediated molecular and phenotypic responses.

The phenotype of a cell and its underlying molecular state is strongly influenced by extracellular signals, including growth factors, hormones, and extracellular matrix proteins. While these signals are normally tightly controlled, their dysregulation leads to phenotypic and molecular states associated with diverse diseases. To develop a detailed understanding of the linkage between molecular and phenotypic changes, we generated a comprehensive dataset that catalogs the transcriptional, proteomic, epigenomic and phenotypic responses of MCF10A mammary epithelial cells after exposure to the ligands EGF, HGF, OSM, IFNG, TGFB and BMP2. Systematic assessment of the molecular and cellular phenotypes induced by these ligands comprise the LINCS Microenvironment (ME) perturbation dataset, which has been curated and made publicly available for community-wide analysis and development of novel computational methods ( synapse.org/LINCS_MCF10A ). In illustrative analyses, we demonstrate how this dataset can be used to discover functionally related molecular features linked to specific cellular phenotypes. Beyond these analyses, this dataset will serve as a resource for the broader scientific community to mine for biological insights, to compare signals carried across distinct molecular modalities, and to develop new computational methods for integrative data analysis.

Integrating multi-omics data reveals function and therapeutic potential of deubiquitinating enzymes.

Deubiquitinating enzymes (DUBs), ~100 of which are found in human cells, are proteases that remove ubiquitin conjugates from proteins, thereby regulating protein turnover. They are involved in a wide range of cellular activities and are emerging therapeutic targets for cancer and other diseases. Drugs targeting USP1 and USP30 are in clinical development for cancer and kidney disease respectively. However, the majority of substrates and pathways regulated by DUBs remain unknown, impeding efforts to prioritize specific enzymes for research and drug development. To assemble a knowledgebase of DUB activities, co-dependent genes, and substrates, we combined targeted experiments using CRISPR libraries and inhibitors with systematic mining of functional genomic databases. Analysis of the Dependency Map, Connectivity Map, Cancer Cell Line Encyclopedia, and multiple protein-protein interaction databases yielded specific hypotheses about DUB function, a subset of which were confirmed in follow-on experiments. The data in this paper are browsable online in a newly developed DUB Portal and promise to improve understanding of DUBs as a family as well as the activities of incompletely characterized DUBs (e.g. USPL1 and USP32) and those already targeted with investigational cancer therapeutics (e.g. USP14, UCHL5, and USP7).

Temporal and spatial topography of cell proliferation in cancer.

Proliferation is a fundamental trait of cancer cells, but its properties and spatial organization in tumours are poorly characterized. Here we use highly multiplexed tissue imaging to perform single-cell quantification of cell cycle regulators and then develop robust, multivariate, proliferation metrics. Across diverse cancers, proliferative architecture is organized at two spatial scales: large domains, and smaller niches enriched for specific immune lineages. Some tumour cells express cell cycle regulators in the (canonical) patterns expected of freely growing cells, a phenomenon we refer to as 'cell cycle coherence'. By contrast, the cell cycles of other tumour cell populations are skewed towards specific phases or exhibit non-canonical (incoherent) marker combinations. Coherence varies across space, with changes in oncogene activity and therapeutic intervention, and is associated with aggressive tumour behaviour. Thus, multivariate measures from high-plex tissue images capture clinically significant features of cancer proliferation, a fundamental step in enabling more precise use of anti-cancer therapies.

The Kinase Chemogenomic Set (KCGS): An Open Science Resource for Kinase Vulnerability Identification.

We describe the assembly and annotation of a chemogenomic set of protein kinase inhibitors as an open science resource for studying kinase biology. The set only includes inhibitors that show potent kinase inhibition and a narrow spectrum of activity when screened across a large panel of kinase biochemical assays. Currently, the set contains 187 inhibitors that cover 215 human kinases. The kinase chemogenomic set (KCGS), current Version 1.0, is the most highly annotated set of selective kinase inhibitors available to researchers for use in cell-based screens.

Heterogeneity and Clonal Evolution of Acquired PARP Inhibitor Resistance in TP53- and BRCA1-Deficient Cells.

Homologous recombination (HR)-deficient cancers are sensitive to poly-ADP ribose polymerase inhibitors (PARPi), which have shown clinical efficacy in the treatment of high-grade serous cancers (HGSC). However, the majority of patients will relapse, and acquired PARPi resistance is emerging as a pressing clinical problem. Here we generated seven single-cell clones with acquired PARPi resistance derived from a PARPi-sensitive TP53 -/- and BRCA1 -/- epithelial cell line generated using CRISPR/Cas9. These clones showed diverse resistance mechanisms, and some clones presented with multiple mechanisms of resistance at the same time. Genomic analysis of the clones revealed unique transcriptional and mutational profiles and increased genomic instability in comparison with a PARPi-sensitive cell line. Clonal evolutionary analyses suggested that acquired PARPi resistance arose via clonal selection from an intrinsically unstable and heterogenous cell population in the sensitive cell line, which contained preexisting drug-tolerant cells. Similarly, clonal and spatial heterogeneity in tumor biopsies from a clinical patient with BRCA1-mutant HGSC with acquired PARPi resistance was observed. In an imaging-based drug screening, the clones showed heterogenous responses to targeted therapeutic agents, indicating that not all PARPi-resistant clones can be targeted with just one therapy. Furthermore, PARPi-resistant clones showed mechanism-dependent vulnerabilities to the selected agents, demonstrating that a deeper understanding on the mechanisms of resistance could lead to improved targeting and biomarkers for HGSC with acquired PARPi resistance. SIGNIFICANCE: This study shows that BRCA1-deficient cells can give rise to multiple genomically and functionally heterogenous PARPi-resistant clones, which are associated with various vulnerabilities that can be targeted in a mechanism-specific manner.

Chemical Biology Toolkit for DCLK1 Reveals Connection to RNA Processing.

Doublecortin-like kinase 1 (DCLK1) is critical for neurogenesis, but overexpression is also observed in multiple cancers and is associated with poor prognosis. Nevertheless, the function of DCLK1 in cancer, especially the context-dependent functions, are poorly understood. We present a "toolkit" that includes the DCLK1 inhibitor DCLK1-IN-1, a complementary DCLK1-IN-1-resistant mutation G532A, and kinase dead mutants D511N and D533N, which can be used to investigate signaling pathways regulated by DCLK1. Using a cancer cell line engineered to be DCLK1 dependent for growth and cell migration, we show that this toolkit can be used to discover associations between DCLK1 kinase activity and biological processes. In particular, we show an association between DCLK1 and RNA processing, including the identification of CDK11 as a potential substrate of DCLK1 using phosphoproteomics.

Development of CDK2 and CDK5 Dual Degrader TMX-2172.

Cyclin-dependent kinase 2 (CDK2) is a potential therapeutic target for the treatment of cancer. Development of CDK2 inhibitors has been extremely challenging as its ATP-binding site shares high similarity with CDK1, a related kinase whose inhibition causes toxic effects. Here, we report the development of TMX-2172, a heterobifunctional CDK2 degrader with degradation selectivity for CDK2 and CDK5 over not only CDK1, but transcriptional CDKs (CDK7 and CDK9) and cell cycle CDKs (CDK4 and CDK6) as well. In addition, we demonstrate that antiproliferative activity in ovarian cancer cells (OVCAR8) depends on CDK2 degradation and correlates with high expression of cyclin E1 (CCNE1), which functions as a regulatory subunit of CDK2. Collectively, our work provides evidence that TMX-2172 represents a lead for further development and that CDK2 degradation is a potentially valuable therapeutic strategy in ovarian and other cancers that overexpress CCNE1.

Inhibition of Haspin Kinase Promotes Cell-Intrinsic and Extrinsic Antitumor Activity.

Patients with melanoma resistant to RAF/MEK inhibitors (RMi) are frequently resistant to other therapies, such as immune checkpoint inhibitors (ICI), and individuals succumb to their disease. New drugs that control tumor growth and favorably modulate the immune environment are therefore needed. We report that the small-molecule CX-6258 has potent activity against both RMi-sensitive (RMS) and -resistant (RMR) melanoma cell lines. Haspin kinase (HASPIN) was identified as a target of CX-6258. HASPIN inhibition resulted in reduced proliferation, frequent formation of micronuclei, recruitment of cGAS, and activation of the cyclic GMP-AMP synthase (cGAS)-stimulator of interferon genes (STING) pathway. In murine models, CX-6258 induced a potent cGAS-dependent type-I IFN response in tumor cells, increased IFNγ-producing CD8+ T cells, and reduced Treg frequency in vivo. HASPIN was more strongly expressed in malignant compared with healthy tissue and its inhibition by CX-6258 had minimal toxicity in ex vivo-expanded human tumor-infiltrating lymphocytes (TIL), proliferating TILs, and in vitro differentiated neurons, suggesting a potential therapeutic index for anticancer therapy. Furthermore, the activity of CX-6258 was validated in several Ewing sarcoma and multiple myeloma cell lines. Thus, HASPIN inhibition may overcome drug resistance in melanoma, modulate the immune environment, and target a vulnerability in different cancer lineages. SIGNIFICANCE: HASPIN inhibition by CX-6258 is a novel and potent strategy for RAF/MEK inhibitor-resistant melanoma and potentially other tumor types. HASPIN inhibition has direct antitumor activity and induces a favorable immune microenvironment.

Development and Characterization of a Wee1 Kinase Degrader.

The G1/S cell cycle checkpoint is frequently dysregulated in cancer, leaving cancer cells reliant on a functional G2/M checkpoint to prevent excessive DNA damage. Wee1 regulates the G2/M checkpoint by phosphorylating CDK1 at Tyr15 to prevent mitotic entry. Previous drug development efforts targeting Wee1 resulted in the clinical-grade inhibitor, AZD1775. However, AZD1775 is burdened by dose-limiting adverse events, and has off-target PLK1 activity. In an attempt to overcome these limitations, we developed Wee1 degraders by conjugating AZD1775 to the cereblon (CRBN)-binding ligand, pomalidomide. The resulting lead compound, ZNL-02-096, degrades Wee1 while sparing PLK1, induces G2/M accumulation at 10-fold lower doses than AZD1775, and synergizes with Olaparib in ovarian cancer cells. We demonstrate that ZNL-02-096 has CRBN-dependent pharmacology that is distinct from AZD1775, which justifies further evaluation of selective Wee1 degraders.

A Multi-center Study on the Reproducibility of Drug-Response Assays in Mammalian Cell Lines.

Evidence that some high-impact biomedical results cannot be repeated has stimulated interest in practices that generate findable, accessible, interoperable, and reusable (FAIR) data. Multiple papers have identified specific examples of irreproducibility, but practical ways to make data more reproducible have not been widely studied. Here, five research centers in the NIH LINCS Program Consortium investigate the reproducibility of a prototypical perturbational assay: quantifying the responsiveness of cultured cells to anti-cancer drugs. Such assays are important for drug development, studying cellular networks, and patient stratification. While many experimental and computational factors impact intra- and inter-center reproducibility, the factors most difficult to identify and control are those with a strong dependency on biological context. These factors often vary in magnitude with the drug being analyzed and with growth conditions. We provide ways to identify such context-sensitive factors, thereby improving both the theory and practice of reproducible cell-based assays.

Multiomics Profiling Establishes the Polypharmacology of FDA-Approved CDK4/6 Inhibitors and the Potential for Differential Clinical Activity.

The target profiles of many drugs are established early in their development and are not systematically revisited at the time of FDA approval. Thus, it is often unclear whether therapeutics with the same nominal targets but different chemical structures are functionally equivalent. In this paper we use five different phenotypic and biochemical assays to compare approved inhibitors of cyclin-dependent kinases 4/6-collectively regarded as breakthroughs in the treatment of hormone receptor-positive breast cancer. We find that transcriptional, proteomic, and phenotypic changes induced by palbociclib, ribociclib, and abemaciclib differ significantly; abemaciclib in particular has advantageous activities partially overlapping those of alvocidib, an older polyselective CDK inhibitor. In cells and mice, abemaciclib inhibits kinases other than CDK4/6 including CDK2/cyclin A/E-implicated in resistance to CDK4/6 inhibition-and CDK1/cyclin B. The multifaceted experimental and computational approaches described here therefore uncover underappreciated differences in CDK4/6 inhibitor activities with potential importance in treating human patients.

Development of a Selective CDK7 Covalent Inhibitor Reveals Predominant Cell-Cycle Phenotype.

Cyclin-dependent kinase 7 (CDK7) regulates both cell cycle and transcription, but its precise role remains elusive. We previously described THZ1, a CDK7 inhibitor, which dramatically inhibits superenhancer-associated gene expression. However, potent CDK12/13 off-target activity obscured CDK7s contribution to this phenotype. Here, we describe the discovery of a highly selective covalent CDK7 inhibitor. YKL-5-124 causes arrest at the G1/S transition and inhibition of E2F-driven gene expression; these effects are rescued by a CDK7 mutant unable to covalently engage YKL-5-124, demonstrating on-target specificity. Unlike THZ1, treatment with YKL-5-124 resulted in no change to RNA polymerase II C-terminal domain phosphorylation; however, inhibition could be reconstituted by combining YKL-5-124 and THZ531, a selective CDK12/13 inhibitor, revealing potential redundancies in CDK control of gene transcription. These findings highlight the importance of CDK7/12/13 polypharmacology for anti-cancer activity of THZ1 and posit that selective inhibition of CDK7 may be useful for treatment of cancers marked by E2F misregulation.

The relative biological effectiveness of low-dose mammography quality X rays in the human breast MCF-10A cell line.

Mammography is used to screen a large fraction of the population for breast cancer, and mammography quality X rays are speculated to be more damaging than the higher energy X rays used for other diagnostic procedures. The radiation dose delivered to breast cells as a result of these screening exposures may be a concern. The purpose of this current study was to determine the relative biological effectiveness (RBE) of low-energy mammography X rays for radiation-induced DNA double-strand breaks evaluated using a highly sensitive automated 53BP1 assay. Automation of the 53BP1 assay enabled the quantification and analysis of meaningful image-based features, including foci counting, within the cell nuclei. Nontumorigenic, human breast epithelial MCF-10A cells were irradiated in the low-dose range with approximately 3-30 mGy of 29 kVp mammography X rays or (137)Cs (662 keV) gamma rays. The induction and resolution of the 53BP1 foci did not differ significantly between exposures to (137)Cs gamma rays and 29 kVp X rays. The RBE was calculated to be 1.1 with a standard deviation of 0.2 for the initial number of radiation-induced double-strand breaks. The radiation dose from a single mammogram did not yield a significant change in the number of detectable foci. However, analysis of additional features revealed subtle differences in the distribution of 53BP1 throughout the nuclei after exposure to the different radiation qualities. A single mammogram was sufficient to alter the distribution of 53BP1 within the nuclear area, but not into discrete foci, while a dose-matched gamma exposure was not sufficient to alter the distribution of 53BP1. Our results indicate that exposure to clinically relevant doses of low-energy mammography quality X rays does not induce more DNA double-strand breaks than exposure to higher energy photons.

A 4 x 500 mm2 cloverleaf detector system for in vivo bone lead measurement.

A 4 x 500 mm2 "cloverleaf" low energy germanium detector array has been assembled for the purpose of in vivo bone lead measurement through x-ray fluorescence. Using 109Cd as an exciting source, results are reported from a leg phantom simulating measurement of lead in a human tibia. For high activity (4.0-4.4 GBq) and low activity (0.18-0.19 GBq) sources, measurement results are reported for both the cloverleaf system and a conventional single detector system of equivalent surface area (2000 mm2). The mean uncertainty and reproducibility of measurement were both significantly improved for the cloverleaf system with a high activity 109Cd source. When using a source activity of 4.4 GBq, measurement of the phantom resulted in an average bone lead uncertainty of 0.79 microg/g and a reproducibility of 0.84 microg/g. These results represent the highest precision yet reported from a bone lead x-ray fluorescence system.