Best practices to evaluate the impact of biomedical research software-metric collection beyond citations.

MOTIVATION: Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation. RESULTS: To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analysing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software's impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress. AVAILABILITY AND IMPLEMENTATION: More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website.

Structure and transcription of integrated HPV DNA in vulvar carcinomas.

HPV infections are associated with a fraction of vulvar cancers. Through hybridization capture and DNA sequencing, HPV DNA was detected in five of thirteen vulvar cancers. HPV16 DNA was integrated into human DNA in three of the five. The insertions were in introns of human NCKAP1, C5orf67, and LRP1B. Integrations in NCKAP1 and C5orf67 were flanked by short direct repeats in the human DNA, consistent with HPV DNA insertions at sites of abortive, staggered, endonucleolytic incisions. The insertion in C5orf67 was present as a 36 kbp, human-HPV-hetero-catemeric DNA as either an extrachromosomal circle or a tandem repeat within the human genome. The human circularization/repeat junction was defined at single nucleotide resolution. The integrated viral DNA segments all retained an intact upstream regulatory region and the adjacent viral E6 and E7 oncogenes. RNA sequencing revealed that the only HPV genes consistently transcribed from the integrated viral DNAs were E7 and E6*I. The other two HPV DNA+ tumors had coinfections, but no evidence for integration. HPV-positive and HPV-negative vulvar cancers exhibited contrasting human, global gene expression patterns partially overlapping with previously observed differences between HPV-positive and HPV-negative cervical and oropharyngeal cancers. A substantial fraction of the differentially expressed genes involved immune system function. Thus, transcription and HPV DNA integration in vulvar cancers resemble those in other HPV-positive cancers. This study emphasizes the power of hybridization capture coupled with DNA and RNA sequencing to identify a broad spectrum of HPV types, determine human genome integration status of viral DNAs, and elucidate their structures.

CTAT-LR-fusion: accurate fusion transcript identification from long and short read isoform sequencing at bulk or single cell resolution.

Gene fusions are found as cancer drivers in diverse adult and pediatric cancers. Accurate detection of fusion transcripts is essential in cancer clinical diagnostics, prognostics, and for guiding therapeutic development. Most currently available methods for fusion transcript detection are compatible with Illumina RNA-seq involving highly accurate short read sequences. Recent advances in long read isoform sequencing enable the detection of fusion transcripts at unprecedented resolution in bulk and single cell samples. Here we developed a new computational tool CTAT-LR-fusion to detect fusion transcripts from long read RNA-seq with or without companion short reads, with applications to bulk or single cell transcriptomes. We demonstrate that CTAT-LR-fusion exceeds fusion detection accuracy of alternative methods as benchmarked with simulated and real long read RNA-seq. Using short and long read RNA-seq, we further apply CTAT-LR-fusion to bulk transcriptomes of nine tumor cell lines, and to tumor single cells derived from a melanoma sample and three metastatic high grade serous ovarian carcinoma samples. In both bulk and in single cell RNA-seq, long isoform reads yielded higher sensitivity for fusion detection than short reads with notable exceptions. By combining short and long reads in CTAT-LR-fusion, we are able to further maximize detection of fusion splicing isoforms and fusion-expressing tumor cells. CTAT-LR-fusion is available at https://github.com/TrinityCTAT/CTAT-LR-fusion/wiki.

De novo detection of somatic variants in long-read single-cell RNA sequencing data.

In cancer, genetic and transcriptomic variations generate clonal heterogeneity, possibly leading to treatment resistance. Long-read single-cell RNA sequencing (LR scRNA-seq) has the potential to detect genetic and transcriptomic variations simultaneously. Here, we present LongSom, a computational workflow leveraging LR scRNA-seq data to call de novo somatic single-nucleotide variants (SNVs), copy-number alterations (CNAs), and gene fusions to reconstruct the tumor clonal heterogeneity. For SNV calling, LongSom distinguishes somatic SNVs from germline polymorphisms by reannotating marker gene expression-based cell types using called variants and applying strict filters. Applying LongSom to ovarian cancer samples, we detected clinically relevant somatic SNVs that were validated against single-cell and bulk panel DNA-seq data and could not be detected with short-read (SR) scRNA-seq. Leveraging somatic SNVs and fusions, LongSom found subclones with different predicted treatment outcomes. In summary, LongSom enables de novo SNVs, CNAs, and fusions detection, thus enabling the study of cancer evolution, clonal heterogeneity, and treatment resistance.

Targeted in silico characterization of fusion transcripts in tumor and normal tissues via FusionInspector.

Here, we present FusionInspector for in silico characterization and interpretation of candidate fusion transcripts from RNA sequencing (RNA-seq) and exploration of their sequence and expression characteristics. We applied FusionInspector to thousands of tumor and normal transcriptomes and identified statistical and experimental features enriched among biologically impactful fusions. Through clustering and machine learning, we identified large collections of fusions potentially relevant to tumor and normal biological processes. We show that biologically relevant fusions are enriched for relatively high expression of the fusion transcript, imbalanced fusion allelic ratios, and canonical splicing patterns, and are deficient in sequence microhomologies between partner genes. We demonstrate that FusionInspector accurately validates fusion transcripts in silico and helps characterize numerous understudied fusions in tumor and normal tissue samples. FusionInspector is freely available as open source for screening, characterization, and visualization of candidate fusions via RNA-seq, and facilitates transparent explanation and interpretation of machine-learning predictions and their experimental sources.

Evaluation of software impact designed for biomedical research: Are we measuring what's meaningful?

Software is vital for the advancement of biology and medicine. Through analysis of usage and impact metrics of software, developers can help determine user and community engagement. These metrics can be used to justify additional funding, encourage additional use, and identify unanticipated use cases. Such analyses can help define improvement areas and assist with managing project resources. However, there are challenges associated with assessing usage and impact, many of which vary widely depending on the type of software being evaluated. These challenges involve issues of distorted, exaggerated, understated, or misleading metrics, as well as ethical and security concerns. More attention to the nuances, challenges, and considerations involved in capturing impact across the diverse spectrum of biological software is needed. Furthermore, some tools may be especially beneficial to a small audience, yet may not have comparatively compelling metrics of high usage. Although some principles are generally applicable, there is not a single perfect metric or approach to effectively evaluate a software tool's impact, as this depends on aspects unique to each tool, how it is used, and how one wishes to evaluate engagement. We propose more broadly applicable guidelines (such as infrastructure that supports the usage of software and the collection of metrics about usage), as well as strategies for various types of software and resources. We also highlight outstanding issues in the field regarding how communities measure or evaluate software impact. To gain a deeper understanding of the issues hindering software evaluations, as well as to determine what appears to be helpful, we performed a survey of participants involved with scientific software projects for the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We also investigated software among this scientific community and others to assess how often infrastructure that supports such evaluations is implemented and how this impacts rates of papers describing usage of the software. We find that although developers recognize the utility of analyzing data related to the impact or usage of their software, they struggle to find the time or funding to support such analyses. We also find that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seem to be associated with increased usage rates. Our findings can help scientific software developers make the most out of the evaluations of their software so that they can more fully benefit from such assessments.

Gene Fusions Create Partner and Collateral Dependencies Essential to Cancer Cell Survival.

Gene fusions frequently result from rearrangements in cancer genomes. In many instances, gene fusions play an important role in oncogenesis; in other instances, they are thought to be passenger events. Although regulatory element rearrangements and copy number alterations resulting from these structural variants are known to lead to transcriptional dysregulation across cancers, the extent to which these events result in functional dependencies with an impact on cancer cell survival is variable. Here we used CRISPR-Cas9 dependency screens to evaluate the fitness impact of 3,277 fusions across 645 cell lines from the Cancer Dependency Map. We found that 35% of cell lines harbored either a fusion partner dependency or a collateral dependency on a gene within the same topologically associating domain as a fusion partner. Fusion-associated dependencies revealed numerous novel oncogenic drivers and clinically translatable alterations. Broadly, fusions can result in partner and collateral dependencies that have biological and clinical relevance across cancer types. SIGNIFICANCE: This study provides insights into how fusions contribute to fitness in different cancer contexts beyond partner-gene activation events, identifying partner and collateral dependencies that may have direct implications for clinical care.

A community challenge to evaluate RNA-seq, fusion detection, and isoform quantification methods for cancer discovery.

The accurate identification and quantitation of RNA isoforms present in the cancer transcriptome is key for analyses ranging from the inference of the impacts of somatic variants to pathway analysis to biomarker development and subtype discovery. The ICGC-TCGA DREAM Somatic Mutation Calling in RNA (SMC-RNA) challenge was a crowd-sourced effort to benchmark methods for RNA isoform quantification and fusion detection from bulk cancer RNA sequencing (RNA-seq) data. It concluded in 2018 with a comparison of 77 fusion detection entries and 65 isoform quantification entries on 51 synthetic tumors and 32 cell lines with spiked-in fusion constructs. We report the entries used to build this benchmark, the leaderboard results, and the experimental features associated with the accurate prediction of RNA species. This challenge required submissions to be in the form of containerized workflows, meaning each of the entries described is easily reusable through CWL and Docker containers at https://github.com/SMC-RNA-challenge. A record of this paper's transparent peer review process is included in the supplemental information.

Opposing immune and genetic mechanisms shape oncogenic programs in synovial sarcoma.

Synovial sarcoma (SyS) is an aggressive neoplasm driven by the SS18-SSX fusion, and is characterized by low T cell infiltration. Here, we studied the cancer-immune interplay in SyS using an integrative approach that combines single-cell RNA sequencing (scRNA-seq), spatial profiling and genetic and pharmacological perturbations. scRNA-seq of 16,872 cells from 12 human SyS tumors uncovered a malignant subpopulation that marks immune-deprived niches in situ and is predictive of poor clinical outcomes in two independent cohorts. Functional analyses revealed that this malignant cell state is controlled by the SS18-SSX fusion, is repressed by cytokines secreted by macrophages and T cells, and can be synergistically targeted with a combination of HDAC and CDK4/CDK6 inhibitors. This drug combination enhanced malignant-cell immunogenicity in SyS models, leading to induced T cell reactivity and T cell-mediated killing. Our study provides a blueprint for investigating heterogeneity in fusion-driven malignancies and demonstrates an interplay between immune evasion and oncogenic processes that can be co-targeted in SyS and potentially in other malignancies.

Imaging-AMARETTO: An Imaging Genomics Software Tool to Interrogate Multiomics Networks for Relevance to Radiography and Histopathology Imaging Biomarkers of Clinical Outcomes.

PURPOSE: The availability of increasing volumes of multiomics, imaging, and clinical data in complex diseases such as cancer opens opportunities for the formulation and development of computational imaging genomics methods that can link multiomics, imaging, and clinical data. METHODS: Here, we present the Imaging-AMARETTO algorithms and software tools to systematically interrogate regulatory networks derived from multiomics data within and across related patient studies for their relevance to radiography and histopathology imaging features predicting clinical outcomes. RESULTS: To demonstrate its utility, we applied Imaging-AMARETTO to integrate three patient studies of brain tumors, specifically, multiomics with radiography imaging data from The Cancer Genome Atlas (TCGA) glioblastoma multiforme (GBM) and low-grade glioma (LGG) cohorts and transcriptomics with histopathology imaging data from the Ivy Glioblastoma Atlas Project (IvyGAP) GBM cohort. Our results show that Imaging-AMARETTO recapitulates known key drivers of tumor-associated microglia and macrophage mechanisms, mediated by STAT3, AHR, and CCR2, and neurodevelopmental and stemness mechanisms, mediated by OLIG2. Imaging-AMARETTO provides interpretation of their underlying molecular mechanisms in light of imaging biomarkers of clinical outcomes and uncovers novel master drivers, THBS1 and MAP2, that establish relationships across these distinct mechanisms. CONCLUSION: Our network-based imaging genomics tools serve as hypothesis generators that facilitate the interrogation of known and uncovering of novel hypotheses for follow-up with experimental validation studies. We anticipate that our Imaging-AMARETTO imaging genomics tools will be useful to the community of biomedical researchers for applications to similar studies of cancer and other complex diseases with available multiomics, imaging, and clinical data.

von Willebrand factor D and EGF domains is an evolutionarily conserved and required feature of blastemas capable of multitissue appendage regeneration.

Regenerative ability varies tremendously across species. A common feature of regeneration of appendages such as limbs, fins, antlers, and tails is the formation of a blastema-a transient structure that houses a pool of progenitor cells that can regenerate the missing tissue. We have identified the expression of von Willebrand factor D and EGF domains (vwde) as a common feature of blastemas capable of regenerating limbs and fins in a variety of highly regenerative species, including axolotl (Ambystoma mexicanum), lungfish (Lepidosiren paradoxa), and Polpyterus (Polypterus senegalus). Further, vwde expression is tightly linked to the ability to regenerate appendages in Xenopus laevis. Functional experiments demonstrate a requirement for vwde in regeneration and indicate that Vwde is a potent growth factor in the blastema. These data identify a key role for vwde in regenerating blastemas and underscore the power of an evolutionarily informed approach for identifying conserved genetic components of regeneration.

Accuracy assessment of fusion transcript detection via read-mapping and de novo fusion transcript assembly-based methods.

BACKGROUND: Accurate fusion transcript detection is essential for comprehensive characterization of cancer transcriptomes. Over the last decade, multiple bioinformatic tools have been developed to predict fusions from RNA-seq, based on either read mapping or de novo fusion transcript assembly. RESULTS: We benchmark 23 different methods including applications we develop, STAR-Fusion and TrinityFusion, leveraging both simulated and real RNA-seq. Overall, STAR-Fusion, Arriba, and STAR-SEQR are the most accurate and fastest for fusion detection on cancer transcriptomes. CONCLUSION: The lower accuracy of de novo assembly-based methods notwithstanding, they are useful for reconstructing fusion isoforms and tumor viruses, both of which are important in cancer research.

Next-generation characterization of the Cancer Cell Line Encyclopedia.

Large panels of comprehensively characterized human cancer models, including the Cancer Cell Line Encyclopedia (CCLE), have provided a rigorous framework with which to study genetic variants, candidate targets, and small-molecule and biological therapeutics and to identify new marker-driven cancer dependencies. To improve our understanding of the molecular features that contribute to cancer phenotypes, including drug responses, here we have expanded the characterizations of cancer cell lines to include genetic, RNA splicing, DNA methylation, histone H3 modification, microRNA expression and reverse-phase protein array data for 1,072 cell lines from individuals of various lineages and ethnicities. Integration of these data with functional characterizations such as drug-sensitivity, short hairpin RNA knockdown and CRISPR-Cas9 knockout data reveals potential targets for cancer drugs and associated biomarkers. Together, this dataset and an accompanying public data portal provide a resource for the acceleration of cancer research using model cancer cell lines.

Influence of Component Geometry on Patellar Mechanics in Posterior-Stabilized Rotating Platform Total Knee Arthroplasty.

BACKGROUND: Patellofemoral complications may cause pain and discomfort, sometimes leading to revision surgery for total knee arthroplasty patients, and patellar implant design has an impact on function of the reconstructed knee. The purpose of this in vivo biomechanics study was to understand the kinematic, functional, strength, and patient-reported outcome data of patients with anatomic and dome patellar implants. METHODS: Satisfactory age-matched, gender-matched, and body mass index-matched patients who underwent rotating-platform total knee arthroplasty from one joint replacement system with either dome (n = 16) or anatomic (n = 16) patellar components were tested in a human motion laboratory using high-speed stereoradiography during an unweighted seated knee extension and a weight-bearing lunge activity. Patellar kinematics, range of motion, strength, and patient-reported outcomes were compared between subjects with anatomic or dome component geometry. RESULTS: Both groups of patients achieved similar functional knee range of motion and reported similar outcomes and satisfaction. On average, patients with the anatomic component had 36% greater extensor strength compared with dome subjects. Patients with anatomic patellar components demonstrated significantly greater flexion of the patella relative to the femur and lower external rotation during the weighted lunge activity. CONCLUSIONS: Relative to the modified dome geometry, patients with anatomic patellar geometry achieved greater patellar flexion which may better replicate normal patellar motion. Patients with anatomic implants may regain more extensor strength compared to patients with dome implants due to geometric differences in the patellar component designs.

Transcriptomic landscape of the blastema niche in regenerating adult axolotl limbs at single-cell resolution.

Regeneration of complex multi-tissue structures, such as limbs, requires the coordinated effort of multiple cell types. In axolotl limb regeneration, the wound epidermis and blastema have been extensively studied via histology, grafting, and bulk-tissue RNA-sequencing. However, defining the contributions of these tissues is hindered due to limited information regarding the molecular identity of the cell types in regenerating limbs. Here we report unbiased single-cell RNA-sequencing on over 25,000 cells from axolotl limbs and identify a plethora of cellular diversity within epidermal, mesenchymal, and hematopoietic lineages in homeostatic and regenerating limbs. We identify regeneration-induced genes, develop putative trajectories for blastema cell differentiation, and propose the molecular identity of fibroblast-like blastema progenitor cells. This work will enable application of molecular techniques to assess the contribution of these populations to limb regeneration. Overall, these data allow for establishment of a putative framework for adult axolotl limb regeneration.

Modified carbazoles destabilize microtubules and kill glioblastoma multiform cells.

Small molecules that target microtubules (MTs) represent promising therapeutics to treat certain types of cancer, including glioblastoma multiform (GBM). We synthesized modified carbazoles and evaluated their antitumor activity in GBM cells in culture. Modified carbazoles with an ethyl moiety linked to the nitrogen of the carbazole and a carbonyl moiety linked to distinct biaromatic rings exhibited remarkably different killing activities in human GBM cell lines and patient-derived GBM cells, with IC50 values from 67 to >10,000 nM. Measures of the activity of modified carbazoles with tubulin and microtubules coupled to molecular docking studies show that these compounds bind to the colchicine site of tubulin in a unique low interaction space that inhibits tubulin assembly. The modified carbazoles reported here represent novel chemical tools to better understand how small molecules disrupt MT functions and kill devastating cancers such as GBM.

Targetable vulnerabilities in T- and NK-cell lymphomas identified through preclinical models.

T- and NK-cell lymphomas (TCL) are a heterogenous group of lymphoid malignancies with poor prognosis. In contrast to B-cell and myeloid malignancies, there are few preclinical models of TCLs, which has hampered the development of effective therapeutics. Here we establish and characterize preclinical models of TCL. We identify multiple vulnerabilities that are targetable with currently available agents (e.g., inhibitors of JAK2 or IKZF1) and demonstrate proof-of-principle for biomarker-driven therapies using patient-derived xenografts (PDXs). We show that MDM2 and MDMX are targetable vulnerabilities within TP53-wild-type TCLs. ALRN-6924, a stapled peptide that blocks interactions between p53 and both MDM2 and MDMX has potent in vitro activity and superior in vivo activity across 8 different PDX models compared to the standard-of-care agent romidepsin. ALRN-6924 induced a complete remission in a patient with TP53-wild-type angioimmunoblastic T-cell lymphoma, demonstrating the potential for rapid translation of discoveries from subtype-specific preclinical models.

Revitalization of a Forward Genetic Screen Identifies Three New Regulators of Fungal Secondary Metabolism in the Genus Aspergillus.

The study of aflatoxin in Aspergillus spp. has garnered the attention of many researchers due to aflatoxin's carcinogenic properties and frequency as a food and feed contaminant. Significant progress has been made by utilizing the model organism Aspergillus nidulans to characterize the regulation of sterigmatocystin (ST), the penultimate precursor of aflatoxin. A previous forward genetic screen identified 23 A. nidulans mutants involved in regulating ST production. Six mutants were characterized from this screen using classical mapping (five mutations in mcsA) and complementation with a cosmid library (one mutation in laeA). The remaining mutants were backcrossed and sequenced using Illumina and Ion Torrent sequencing platforms. All but one mutant contained one or more sequence variants in predicted open reading frames. Deletion of these genes resulted in identification of mutant alleles responsible for the loss of ST production in 12 of the 17 remaining mutants. Eight of these mutations were in genes already known to affect ST synthesis (laeA, mcsA, fluG, and stcA), while the remaining four mutations (in laeB, sntB, and hamI) were in previously uncharacterized genes not known to be involved in ST production. Deletion of laeB, sntB, and hamI in A. flavus results in loss of aflatoxin production, confirming that these regulators are conserved in the aflatoxigenic aspergilli. This report highlights the multifaceted regulatory mechanisms governing secondary metabolism in Aspergillus Additionally, these data contribute to the increasing number of studies showing that forward genetic screens of fungi coupled with whole-genome resequencing is a robust and cost-effective technique.IMPORTANCE In a postgenomic world, reverse genetic approaches have displaced their forward genetic counterparts. The techniques used in forward genetics to identify loci of interest were typically very cumbersome and time-consuming, relying on Mendelian traits in model organisms. The current work was pursued not only to identify alleles involved in regulation of secondary metabolism but also to demonstrate a return to forward genetics to track phenotypes and to discover genetic pathways that could not be predicted through a reverse genetics approach. While identification of mutant alleles from whole-genome sequencing has been done before, here we illustrate the possibility of coupling this strategy with a genetic screen to identify multiple alleles of interest. Sequencing of classically derived mutants revealed several uncharacterized genes, which represent novel pathways to regulate and control the biosynthesis of sterigmatocystin and of aflatoxin, a societally and medically important mycotoxin.

Corrigendum: Contribution of classical end-joining to PTEN inactivation in p53-mediated glioblastoma formation and drug-resistant survival.

This corrects the article DOI: 10.1038/ncomms14013.