Text summarization with ChatGPT for drug labeling documents.

Text summarization is crucial in scientific research, drug discovery and development, regulatory review, and more. This task demands domain expertise, language proficiency, semantic prowess, and conceptual skill. The recent advent of large language models (LLMs), such as ChatGPT, offers unprecedented opportunities to automate this process. We compared ChatGPT-generated summaries with those produced by human experts using FDA drug labeling documents. The labeling contains summaries of key labeling sections, making them an ideal human benchmark to evaluate ChatGPT's summarization capabilities. Analyzing >14000 summaries, we observed that ChatGPT-generated summaries closely resembled those generated by human experts. Importantly, ChatGPT exhibited even greater similarity when summarizing drug safety information. These findings highlight ChatGPT's potential to accelerate work in critical areas, including drug safety.

Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA.

Accurately calling indels with next-generation sequencing (NGS) data is critical for clinical application. The precisionFDA team collaborated with the U.S. Food and Drug Administration's (FDA's) National Center for Toxicological Research (NCTR) and successfully completed the NCTR Indel Calling from Oncopanel Sequencing Data Challenge, to evaluate the performance of indel calling pipelines. Top performers were selected based on precision, recall, and F1-score. The performance of many other pipelines was close to the top performers, which produced a top cluster of performers. The performance was significantly higher in high confidence regions and coding regions, and significantly lower in low complexity regions. Oncopanel capture and other issues may have occurred that affected the recall rate. Indels with higher variant allele frequency (VAF) may generally be called with higher confidence. Many of the indel calling pipelines had good performance. Some of them performed generally well across all three oncopanels, while others were better for a specific oncopanel. The performance of indel calling can further be improved by restricting the calls within high confidence intervals (HCIs) and coding regions, and by excluding low complexity regions (LCR) regions. Certain VAF cut-offs could be applied according to the applications.

Extend the benchmarking indel set by manual review using the individual cell line sequencing data from the Sequencing Quality Control 2 (SEQC2) project.

Accurate indel calling plays an important role in precision medicine. A benchmarking indel set is essential for thoroughly evaluating the indel calling performance of bioinformatics pipelines. A reference sample with a set of known-positive variants was developed in the FDA-led Sequencing Quality Control Phase 2 (SEQC2) project, but the known indels in the known-positive set were limited. This project sought to provide an enriched set of known indels that would be more translationally relevant by focusing on additional cancer related regions. A thorough manual review process completed by 42 reviewers, two advisors, and a judging panel of three researchers significantly enriched the known indel set by an additional 516 indels. The extended benchmarking indel set has a large range of variant allele frequencies (VAFs), with 87% of them having a VAF below 20% in reference Sample A. The reference Sample A and the indel set can be used for comprehensive benchmarking of indel calling across a wider range of VAF values in the lower range. Indel length was also variable, but the majority were under 10 base pairs (bps). Most of the indels were within coding regions, with the remainder in the gene regulatory regions. Although high confidence can be derived from the robust study design and meticulous human review, this extensive indel set has not undergone orthogonal validation. The extended benchmarking indel set, along with the indels in the previously published known-positive set, was the truth set used to benchmark indel calling pipelines in a community challenge hosted on the precisionFDA platform. This benchmarking indel set and reference samples can be utilized for a comprehensive evaluation of indel calling pipelines. Additionally, the insights and solutions obtained during the manual review process can aid in improving the performance of these pipelines.

Structural variant analysis of a cancer reference cell line sample using multiple sequencing technologies.

BACKGROUND: The cancer genome is commonly altered with thousands of structural rearrangements including insertions, deletions, translocation, inversions, duplications, and copy number variations. Thus, structural variant (SV) characterization plays a paramount role in cancer target identification, oncology diagnostics, and personalized medicine. As part of the SEQC2 Consortium effort, the present study established and evaluated a consensus SV call set using a breast cancer reference cell line and matched normal control derived from the same donor, which were used in our companion benchmarking studies as reference samples. RESULTS: We systematically investigated somatic SVs in the reference cancer cell line by comparing to a matched normal cell line using multiple NGS platforms including Illumina short-read, 10X Genomics linked reads, PacBio long reads, Oxford Nanopore long reads, and high-throughput chromosome conformation capture (Hi-C). We established a consensus SV call set of a total of 1788 SVs including 717 deletions, 230 duplications, 551 insertions, 133 inversions, 146 translocations, and 11 breakends for the reference cancer cell line. To independently evaluate and cross-validate the accuracy of our consensus SV call set, we used orthogonal methods including PCR-based validation, Affymetrix arrays, Bionano optical mapping, and identification of fusion genes detected from RNA-seq. We evaluated the strengths and weaknesses of each NGS technology for SV determination, and our findings provide an actionable guide to improve cancer genome SV detection sensitivity and accuracy. CONCLUSIONS: A high-confidence consensus SV call set was established for the reference cancer cell line. A large subset of the variants identified was validated by multiple orthogonal methods.

Deep oncopanel sequencing reveals within block position-dependent quality degradation in FFPE processed samples.

BACKGROUND: Clinical laboratories routinely use formalin-fixed paraffin-embedded (FFPE) tissue or cell block cytology samples in oncology panel sequencing to identify mutations that can predict patient response to targeted therapy. To understand the technical error due to FFPE processing, a robustly characterized diploid cell line was used to create FFPE samples with four different pre-tissue processing formalin fixation times. A total of 96 FFPE sections were then distributed to different laboratories for targeted sequencing analysis by four oncopanels, and variants resulting from technical error were identified. RESULTS: Tissue sections that fail more frequently show low cellularity, lower than recommended library preparation DNA input, or target sequencing depth. Importantly, sections from block surfaces are more likely to show FFPE-specific errors, akin to "edge effects" seen in histology, while the inner samples display no quality degradation related to fixation time. CONCLUSIONS: To assure reliable results, we recommend avoiding the block surface portion and restricting mutation detection to genomic regions of high confidence.

Ultra-deep multi-oncopanel sequencing of benchmarking samples with a wide range of variant allele frequencies.

The lack of suitable reference genomic material to enable a transparent cross-lab study of oncopanels inspired the SEQC2 Oncopanel Sequencing Working Group to develop four reference samples, sequenced with eight oncopanels at independent test laboratories with ultra-deep sequencing depth. This rich, publicly available dataset enabled performance assessment of the clinical applicability of oncopanels. In addition, this dataset present sample opportunities for developing specific and robust bioinformatics pipelines and fine-tuning parameters for more accurate variant calling, investigating ideal sequencing depth for variant calling of a given minimum VAF and variant type, and also recommending best use cases for Unique Molecular Identifier (UMI) technology.

Assessing reproducibility of inherited variants detected with short-read whole genome sequencing.

BACKGROUND: Reproducible detection of inherited variants with whole genome sequencing (WGS) is vital for the implementation of precision medicine and is a complicated process in which each step affects variant call quality. Systematically assessing reproducibility of inherited variants with WGS and impact of each step in the process is needed for understanding and improving quality of inherited variants from WGS. RESULTS: To dissect the impact of factors involved in detection of inherited variants with WGS, we sequence triplicates of eight DNA samples representing two populations on three short-read sequencing platforms using three library kits in six labs and call variants with 56 combinations of aligners and callers. We find that bioinformatics pipelines (callers and aligners) have a larger impact on variant reproducibility than WGS platform or library preparation. Single-nucleotide variants (SNVs), particularly outside difficult-to-map regions, are more reproducible than small insertions and deletions (indels), which are least reproducible when > 5 bp. Increasing sequencing coverage improves indel reproducibility but has limited impact on SNVs above 30×. CONCLUSIONS: Our findings highlight sources of variability in variant detection and the need for improvement of bioinformatics pipelines in the era of precision medicine with WGS.

Orchestrating and sharing large multimodal data for transparent and reproducible research.

Reproducibility is essential to open science, as there is limited relevance for findings that can not be reproduced by independent research groups, regardless of its validity. It is therefore crucial for scientists to describe their experiments in sufficient detail so they can be reproduced, scrutinized, challenged, and built upon. However, the intrinsic complexity and continuous growth of biomedical data makes it increasingly difficult to process, analyze, and share with the community in a FAIR (findable, accessible, interoperable, and reusable) manner. To overcome these issues, we created a cloud-based platform called ORCESTRA ( orcestra.ca ), which provides a flexible framework for the reproducible processing of multimodal biomedical data. It enables processing of clinical, genomic and perturbation profiles of cancer samples through automated processing pipelines that are user-customizable. ORCESTRA creates integrated and fully documented data objects with persistent identifiers (DOI) and manages multiple dataset versions, which can be shared for future studies.

Clitoral Priapism in a Transgender Male.

INTRODUCTION: Clitoral priapism due to venous outflow obstruction is a rare event and medical emergency. Androgen-induced clitoromegaly in transgender men has not been previously identified as a risk factor. AIMS: Advance current knowledge on identification and treatment of clitoral priapism in the transgender male. METHODS: A 32 year-old presurgical transgender male underwent gender-affirming laparoscopic total hysterectomy and bilateral salpingo-oöphorectomy without incident. Seven days postop, he developed progressive and painful clitoral engorgement that was persistent. Examination and imaging were consistent with clitoral priapism. RESULTS: Clitoral priapism was treated with adrenergic drugs (imipramine and pseudoephedrine) with rapid resolution of symptoms. CONCLUSION: Clitoral priapism is a rare phenomenon usually associated with use of a psychotropic medication. Whether clitoromegaly secondary to androgen administration in transgender men is a risk factor for this rare medical emergency is unknown. Prompt recognition and treatment is paramount. Kusko RE, Singhal E, Kauffman RP. Clitoral Priapism in a Transgender Male. Sex Med 2021;9:100431.

Toward best practice in cancer mutation detection with whole-genome and whole-exome sequencing.

Clinical applications of precision oncology require accurate tests that can distinguish true cancer-specific mutations from errors introduced at each step of next-generation sequencing (NGS). To date, no bulk sequencing study has addressed the effects of cross-site reproducibility, nor the biological, technical and computational factors that influence variant identification. Here we report a systematic interrogation of somatic mutations in paired tumor-normal cell lines to identify factors affecting detection reproducibility and accuracy at six different centers. Using whole-genome sequencing (WGS) and whole-exome sequencing (WES), we evaluated the reproducibility of different sample types with varying input amount and tumor purity, and multiple library construction protocols, followed by processing with nine bioinformatics pipelines. We found that read coverage and callers affected both WGS and WES reproducibility, but WES performance was influenced by insert fragment size, genomic copy content and the global imbalance score (GIV; G > T/C > A). Finally, taking into account library preparation protocol, tumor content, read coverage and bioinformatics processes concomitantly, we recommend actionable practices to improve the reproducibility and accuracy of NGS experiments for cancer mutation detection.

Cross-oncopanel study reveals high sensitivity and accuracy with overall analytical performance depending on genomic regions.

BACKGROUND: Targeted sequencing using oncopanels requires comprehensive assessments of accuracy and detection sensitivity to ensure analytical validity. By employing reference materials characterized by the U.S. Food and Drug Administration-led SEquence Quality Control project phase2 (SEQC2) effort, we perform a cross-platform multi-lab evaluation of eight Pan-Cancer panels to assess best practices for oncopanel sequencing. RESULTS: All panels demonstrate high sensitivity across targeted high-confidence coding regions and variant types for the variants previously verified to have variant allele frequency (VAF) in the 5-20% range. Sensitivity is reduced by utilizing VAF thresholds due to inherent variability in VAF measurements. Enforcing a VAF threshold for reporting has a positive impact on reducing false positive calls. Importantly, the false positive rate is found to be significantly higher outside the high-confidence coding regions, resulting in lower reproducibility. Thus, region restriction and VAF thresholds lead to low relative technical variability in estimating promising biomarkers and tumor mutational burden. CONCLUSION: This comprehensive study provides actionable guidelines for oncopanel sequencing and clear evidence that supports a simplified approach to assess the analytical performance of oncopanels. It will facilitate the rapid implementation, validation, and quality control of oncopanels in clinical use.

A verified genomic reference sample for assessing performance of cancer panels detecting small variants of low allele frequency.

BACKGROUND: Oncopanel genomic testing, which identifies important somatic variants, is increasingly common in medical practice and especially in clinical trials. Currently, there is a paucity of reliable genomic reference samples having a suitably large number of pre-identified variants for properly assessing oncopanel assay analytical quality and performance. The FDA-led Sequencing and Quality Control Phase 2 (SEQC2) consortium analyze ten diverse cancer cell lines individually and their pool, termed Sample A, to develop a reference sample with suitably large numbers of coding positions with known (variant) positives and negatives for properly evaluating oncopanel analytical performance. RESULTS: In reference Sample A, we identify more than 40,000 variants down to 1% allele frequency with more than 25,000 variants having less than 20% allele frequency with 1653 variants in COSMIC-related genes. This is 5-100× more than existing commercially available samples. We also identify an unprecedented number of negative positions in coding regions, allowing statistical rigor in assessing limit-of-detection, sensitivity, and precision. Over 300 loci are randomly selected and independently verified via droplet digital PCR with 100% concordance. Agilent normal reference Sample B can be admixed with Sample A to create new samples with a similar number of known variants at much lower allele frequency than what exists in Sample A natively, including known variants having allele frequency of 0.02%, a range suitable for assessing liquid biopsy panels. CONCLUSION: These new reference samples and their admixtures provide superior capability for performing oncopanel quality control, analytical accuracy, and validation for small to large oncopanels and liquid biopsy assays.

A primer on applying AI synergistically with domain expertise to oncology.

BACKGROUND: The concurrent growth of large-scale oncology data alongside the computational methods with which to analyze and model it has created a promising environment for revolutionizing cancer diagnosis, treatment, prevention, and drug discovery. Computational methods applied to large datasets have accelerated the drug discovery process by reducing bottlenecks and widening the search space beyond what is experimentally tractable. As the research community gains understanding of the myriad genetic underpinnings of cancer via sequencing, imaging, screens, and more that are ingested, transformed, and modeled by top open-source machine learning and artificial intelligence tools readily available, the next big drug candidate might seem merely an "Enter" key away. Of course, the reality is more convoluted, but still promising. SCOPE OF REVIEW: We present methods to approach the process of building an AI model, with strong emphasis on the aspects of model development we believe to be crucial to success but that are not commonly discussed: diligence in posing questions, identifying suitable datasets and curating them, and collaborating closely with biology and oncology experts while designing and evaluating the model. Digital pathology, Electronic Health Records, and other data types outside of high-throughput molecular data are reviewed well by others and outside of the scope of this review. This review emphasizes the importance of considering the limitations of the datasets, computational methods, and our minds when designing AI models. For example, datasets can be biased towards areas of research interest, funding, and particular patient populations. Neural networks may learn representations and correlations within the data that are grounded not in biological phenomena, but statistical anomalies erroneously extracted from the training data. Researchers may mis-interpret or over-interpret the output, or design and evaluate the training process such that the resultant model generalizes poorly. Fortunately, awareness of the strengths and limitations of applying data analytics and AI to drug discovery enables us to leverage them carefully and insightfully while maximizing their utility. These applications when performed in close collaboration with domain experts, together with continuous critical evaluation, generation of new data to minimize known blind spots as they are found, and rigorous experimental validation, increases the success rate of the study. We will discuss applications including AI-assisted target identification, drug repurposing, patient stratification, and gene prioritization. MAJOR CONCLUSIONS: Data analytics and AI have demonstrated capabilities to revolutionize cancer research, prevention, and treatment by maximizing our understanding and use of the expanding panoply of experimental data. However, to separate promise from true utility, computational tools must be carefully designed, critically evaluated, and constantly improved. Once that is achieved, a human-computer hybrid discovery process will outperform one driven by each alone. GENERAL SIGNIFICANCE: This review highlights the challenges and promise of synergizing predictive AI models with human expertise towards greater understanding of cancer.

Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology.

Circulating tumor DNA (ctDNA) sequencing is being rapidly adopted in precision oncology, but the accuracy, sensitivity and reproducibility of ctDNA assays is poorly understood. Here we report the findings of a multi-site, cross-platform evaluation of the analytical performance of five industry-leading ctDNA assays. We evaluated each stage of the ctDNA sequencing workflow with simulations, synthetic DNA spike-in experiments and proficiency testing on standardized, cell-line-derived reference samples. Above 0.5% variant allele frequency, ctDNA mutations were detected with high sensitivity, precision and reproducibility by all five assays, whereas, below this limit, detection became unreliable and varied widely between assays, especially when input material was limited. Missed mutations (false negatives) were more common than erroneous candidates (false positives), indicating that the reliable sampling of rare ctDNA fragments is the key challenge for ctDNA assays. This comprehensive evaluation of the analytical performance of ctDNA assays serves to inform best practice guidelines and provides a resource for precision oncology.

Advancing NGS quality control to enable measurement of actionable mutations in circulating tumor DNA.

The primary objective of the FDA-led Sequencing and Quality Control Phase 2 (SEQC2) project is to develop standard analysis protocols and quality control metrics for use in DNA testing to enhance scientific research and precision medicine. This study reports a targeted next-generation sequencing (NGS) method that will enable more accurate detection of actionable mutations in circulating tumor DNA (ctDNA) clinical specimens. To accomplish this, a synthetic internal standard spike-in was designed for each actionable mutation target, suitable for use in NGS following hybrid capture enrichment and unique molecular index (UMI) or non-UMI library preparation. When mixed with contrived ctDNA reference samples, internal standards enabled calculation of technical error rate, limit of blank, and limit of detection for each variant at each nucleotide position in each sample. True-positive mutations with variant allele fraction too low for detection by current practice were detected with this method, thereby increasing sensitivity.

FDA-led consortium studies advance quality control of targeted next generation sequencing assays for precision oncology.

Cancer is the second leading cause of mortality worldwide despite tremendous advances in treatment. The promise of precision oncology depends on accurate characterization of tumor mutations and subsequent therapy selection. The lack of tumor reference samples along with the associated next generation sequencing (NGS) technical assessments has hindered the development of NGS assays and the realization of benefits for precision oncology. The summarized results and recommendations of several seminal SEQC2 studies along with a vision of the changing landscape of precision oncology and anticipated next steps by the SEQC2 consortium are reported. Importantly, these studies utilized a new robust reference sample material which was developed and constructed to support multiple DNA and RNA-based NGS assay studies. These studies focused on a wide variety of precision oncology assay scenarios and provided guidelines for standardized analyses and best practice recommendations. The evolving landscape of precision oncology requires insights into critical factors supporting the sensitivity and reproducibility of clinical NGS assays for continued improvement in patient outcomes. Persistent development of robust reference materials, quantitative performance metrics, and actionable data analysis recommendations are needed. This series of SEQC2 studies serve to advance NGS-based assays for precision oncology and support regulatory science endeavors.

pS421 huntingtin modulates mitochondrial phenotypes and confers neuroprotection in an HD hiPSC model.

Huntington disease (HD) is a hereditary neurodegenerative disorder caused by mutant huntingtin (mHTT). Phosphorylation at serine-421 (pS421) of mHTT has been shown to be neuroprotective in cellular and rodent models. However, the genetic context of these models differs from that of HD patients. Here we employed human pluripotent stem cells (hiPSCs), which express endogenous full-length mHTT. Using genome editing, we generated isogenic hiPSC lines in which the S421 site in mHTT has been mutated into a phospho-mimetic aspartic acid (S421D) or phospho-resistant alanine (S421A). We observed that S421D, rather than S421A, confers neuroprotection in hiPSC-derived neural cells. Although we observed no effect of S421D on mHTT clearance or axonal transport, two aspects previously reported to be impacted by phosphorylation of mHTT at S421, our analysis revealed modulation of several aspects of mitochondrial form and function. These include mitochondrial surface area, volume, and counts, as well as improved mitochondrial membrane potential and oxidative phosphorylation. Our study validates the protective role of pS421 on mHTT and highlights a facet of the relationship between mHTT and mitochondrial changes in the context of human physiology with potential relevance to the pathogenesis of HD.

Quantitative Structure-Activity Relationship Models for Predicting Inflammatory Potential of Metal Oxide Nanoparticles.

BACKGROUND: Although substantial concerns about the inflammatory effects of engineered nanomaterial (ENM) have been raised, experimentally assessing toxicity of various ENMs is challenging and time-consuming. Alternatively, quantitative structure-activity relationship (QSAR) models have been employed to assess nanosafety. However, no previous attempt has been made to predict the inflammatory potential of ENMs. OBJECTIVES: By employing metal oxide nanoparticles (MeONPs) as a model ENM, we aimed to develop QSAR models for prediction of the inflammatory potential by their physicochemical properties. METHODS: We built a comprehensive data set of 30 MeONPs to screen a proinflammatory cytokine interleukin (IL)-1 beta (IL-1ß) release in THP-1 cell line. The in vitro hazard ranking was validated in mouse lungs by oropharyngeal instillation of six randomly selected MeONPs. We established QSAR models for prediction of MeONP-induced inflammatory potential via machine learning. The models were further validated against seven new MeONPs. Density functional theory (DFT) computations were exploited to decipher the key mechanisms driving inflammatory responses of MeONPs. RESULTS: Seventeen out of 30 MeONPs induced excess IL-1ß production in THP-1 cells. In vivo disease outcomes were highly relevant to the in vitro data. QSAR models were developed for inflammatory potential, with predictive accuracy (ACC) exceeding 90%. The models were further validated experimentally against seven independent MeONPs (ACC=86%). DFT computations and experimental results further revealed the underlying mechanisms: MeONPs with metal electronegativity lower than 1.55 and positive ζ-potential were more likely to cause lysosomal damage and inflammation. CONCLUSIONS: IL-1ß released in THP-1 cells can be an index to rank the inflammatory potential of MeONPs. QSAR models based on IL-1ß were able to predict the inflammatory potential of MeONPs. Our approach overcame the challenge of time- and labor-consuming biological experiments and allowed for computational assessment of MeONP inflammatory potential by characterization of their physicochemical properties. https://doi.org/10.1289/EHP6508.

Advanced bioinformatics rapidly identifies existing therapeutics for patients with coronavirus disease-2019 (COVID-19).

BACKGROUND: The recent global pandemic has placed a high priority on identifying drugs to prevent or lessen clinical infection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), caused by Coronavirus disease-2019 (COVID-19). METHODS: We applied two computational approaches to identify potential therapeutics. First, we sought to identify existing FDA approved drugs that could block coronaviruses from entering cells by binding to ACE2 or TMPRSS2 using a high-throughput AI-based binding affinity prediction platform. Second, we sought to identify FDA approved drugs that could attenuate the gene expression patterns induced by coronaviruses, using our Disease Cancelling Technology (DCT) platform. RESULTS: Top results for ACE2 binding iincluded several ACE inhibitors, a beta-lactam antibiotic, two antiviral agents (Fosamprenavir and Emricasan) and glutathione. The platform also assessed specificity for ACE2 over ACE1, important for avoiding counterregulatory effects. Further studies are needed to weigh the benefit of blocking virus entry against potential counterregulatory effects and possible protective effects of ACE2. However, the data herein suggest readily available drugs that warrant experimental evaluation to assess potential benefit. DCT was run on an animal model of SARS-CoV, and ranked compounds by their ability to induce gene expression signals that counteract disease-associated signals. Top hits included Vitamin E, ruxolitinib, and glutamine. Glutathione and its precursor glutamine were highly ranked by two independent methods, suggesting both warrant further investigation for potential benefit against SARS-CoV-2. CONCLUSIONS: While these findings are not yet ready for clinical translation, this report highlights the potential use of two bioinformatics technologies to rapidly discover existing therapeutic agents that warrant further investigation for established and emerging disease processes.

Correction to: Similarities and differences between variants called with human reference genome HG19 or HG38.

After publication of this supplement article.