RESUMO
PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text articles from the PMC open access subset, updated weekly. PubTator 3.0's online interface and API utilize these precomputed entity relations and synonyms to provide advanced search capabilities and enable large-scale analyses, streamlining many complex information needs. We showcase the retrieval quality of PubTator 3.0 using a series of entity pair queries, demonstrating that PubTator 3.0 retrieves a greater number of articles than either PubMed or Google Scholar, with higher precision in the top 20 results. We further show that integrating ChatGPT (GPT-4) with PubTator APIs dramatically improves the factuality and verifiability of its responses. In summary, PubTator 3.0 offers a comprehensive set of features and tools that allow researchers to navigate the ever-expanding wealth of biomedical literature, expediting research and unlocking valuable insights for scientific discovery.
Assuntos
PubMed , Inteligência Artificial , Humanos , Software , Mineração de Dados/métodos , Semântica , InternetRESUMO
ChatGPT has drawn considerable attention from both the general public and domain experts with its remarkable text generation capabilities. This has subsequently led to the emergence of diverse applications in the field of biomedicine and health. In this work, we examine the diverse applications of large language models (LLMs), such as ChatGPT, in biomedicine and health. Specifically, we explore the areas of biomedical information retrieval, question answering, medical text summarization, information extraction and medical education and investigate whether LLMs possess the transformative power to revolutionize these tasks or whether the distinct complexities of biomedical domain presents unique challenges. Following an extensive literature survey, we find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods. For other applications, the advances have been modest. Overall, LLMs have not yet revolutionized biomedicine, but recent rapid progress indicates that such methods hold great potential to provide valuable means for accelerating discovery and improving health. We also find that the use of LLMs, like ChatGPT, in the fields of biomedicine and health entails various risks and challenges, including fabricated information in its generated responses, as well as legal and privacy concerns associated with sensitive patient data. We believe this survey can provide a comprehensive and timely overview to biomedical researchers and healthcare practitioners on the opportunities and challenges associated with using ChatGPT and other LLMs for transforming biomedicine and health.
Assuntos
Armazenamento e Recuperação da Informação , Idioma , Humanos , Privacidade , PesquisadoresRESUMO
BACKGROUND: Although patients have easy access to their electronic health records and laboratory test result data through patient portals, laboratory test results are often confusing and hard to understand. Many patients turn to web-based forums or question-and-answer (Q&A) sites to seek advice from their peers. The quality of answers from social Q&A sites on health-related questions varies significantly, and not all responses are accurate or reliable. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to have their questions answered. OBJECTIVE: We aimed to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to laboratory test-related questions asked by patients and identify potential issues that can be mitigated using augmentation approaches. METHODS: We collected laboratory test result-related Q&A data from Yahoo! Answers and selected 53 Q&A pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from 5 LLMs: GPT-4, GPT-3.5, LLaMA 2, MedAlpaca, and ORCA_mini. We assessed the similarity of their answers using standard Q&A similarity-based evaluation metrics, including Recall-Oriented Understudy for Gisting Evaluation, Bilingual Evaluation Understudy, Metric for Evaluation of Translation With Explicit Ordering, and Bidirectional Encoder Representations from Transformers Score. We used an LLM-based evaluator to judge whether a target model had higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. We performed a manual evaluation with medical experts for all the responses to 7 selected questions on the same 4 aspects. RESULTS: Regarding the similarity of the responses from 4 LLMs; the GPT-4 output was used as the reference answer, the responses from GPT-3.5 were the most similar, followed by those from LLaMA 2, ORCA_mini, and MedAlpaca. Human answers from Yahoo data were scored the lowest and, thus, as the least similar to GPT-4-generated answers. The results of the win rate and medical expert evaluation both showed that GPT-4's responses achieved better scores than all the other LLM responses and human responses on all 4 aspects (relevance, correctness, helpfulness, and safety). LLM responses occasionally also suffered from lack of interpretation in one's medical context, incorrect statements, and lack of references. CONCLUSIONS: By evaluating LLMs in generating responses to patients' laboratory test result-related questions, we found that, compared to other 4 LLMs and human answers from a Q&A website, GPT-4's responses were more accurate, helpful, relevant, and safer. There were cases in which GPT-4 responses were inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses, including prompt engineering, prompt augmentation, retrieval-augmented generation, and response evaluation.
Assuntos
Inteligência Artificial , Registros Eletrônicos de Saúde , Humanos , IdiomaRESUMO
Atomically precise supported nanocluster catalysts (APSNCs), which feature exact atomic composition, well-defined structures, and unique catalytic properties, offer an exceptional platform for understanding the structure-performance relationship at the atomic level. However, fabricating APSNCs with precisely controlled and uniform metal atom numbers, as well as maintaining a stable structure, remains a significant challenge due to uncontrollable dispersion and easy aggregation during synthetic and catalytic processes. Herein, we developed an effective ligand engineering strategy to construct a Pt6 nanocluster catalyst stabilized on oxidized carbon nanotubes (Pt6/OCNT). The structural analysis revealed that Pt6 nanoclusters in Pt6/OCNT were fully exposed and exhibited a planar structure. Furthermore, the obtained Pt6/OCNT exhibited outstanding acidic HOR performances with a high mass activity of 18.37 A·mgpt-1 along with excellent stability during a 24 h constant operation and good CO tolerance, surpassing those of the commercial Pt/C. Density functional theory (DFT) calculations demonstrated that the unique geometric and electronic structures of Pt6 nanoclusters on OCNT altered the hydrogen adsorption energies on catalytic sites and thus lowered the HOR theoretical overpotential. This work presents a new prospect for designing and synthesizing advanced APSNCs for efficient energy electrocatalysis.
RESUMO
Lead Pb(II) ions is a cumulative toxicant that impacts several biological systems and poses severe harm to young children. Accurate Pb(II) ions monitoring is thus of paramount importance. Here, we present the synthesis and application of glutathione-capped Au15 nanoclusters (Au15(SG)13) as a luminescence probe for the accurate and selective monitoring of blood Pb(II). The introduction of Pb(II) ions triggers orderly self-assembly of Au15 nanoclusters, resulting in the formation of rigid shell around Au nuclei. This limits the localized vibration of the glutathione ligands and their interaction with water molecules, greatly reducing non-radiative energy loss, and thereby enhancing the photoluminescence signal. Consequently, Au15(SG)13 nanoclusters exhibit high sensitivity for Pb(II) detection. The detection signal displays a linear relationship with Pb(II) over a wide detection range (0-800 µg/L), demonstrating a substantial sensitivity of 35.29 µg/L. Moreover, the developed nanoclusters show superior selectivity for Pb(II) ions, distinguishing them from other prevalent heavy metals. This work pave the way for the development of advanced Pb(II) sensors with high sensitivity and selectivity.
Assuntos
Luminescência , Nanopartículas Metálicas , Criança , Humanos , Pré-Escolar , Chumbo , Ligantes , Íons , Glutationa , OuroRESUMO
Gene set knowledge discovery is essential for advancing human functional genomics. Recent studies have shown promising performance by harnessing the power of Large Language Models (LLMs) on this task. Nonetheless, their results are subject to several limitations common in LLMs such as hallucinations. In response, we present GeneAgent, a first-of-its-kind language agent featuring self-verification capability. It autonomously interacts with various biological databases and leverages relevant domain knowledge to improve accuracy and reduce hallucination occurrences. Benchmarking on 1,106 gene sets from different sources, GeneAgent consistently outperforms standard GPT-4 by a significant margin. Moreover, a detailed manual review confirms the effectiveness of the self-verification module in minimizing hallucinations and generating more reliable analytical narratives. To demonstrate its practical utility, we apply GeneAgent to seven novel gene sets derived from mouse B2905 melanoma cell lines, with expert evaluations showing that GeneAgent offers novel insights into gene functions and subsequently expedites knowledge discovery.
RESUMO
PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases, and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text articles from the PMC open access subset, updated weekly. PubTator 3.0's online interface and API utilize these precomputed entity relations and synonyms to provide advanced search capabilities and enable large-scale analyses, streamlining many complex information needs. We showcase the retrieval quality of PubTator 3.0 using a series of entity pair queries, demonstrating that PubTator 3.0 retrieves a greater number of articles than either PubMed or Google Scholar, with higher precision in the top 20 results. We further show that integrating ChatGPT (GPT-4) with PubTator APIs dramatically improves the factuality and verifiability of its responses. In summary, PubTator 3.0 offers a comprehensive set of features and tools that allow researchers to navigate the ever-expanding wealth of biomedical literature, expediting research and unlocking valuable insights for scientific discovery.
RESUMO
BACKGROUND: Rectal cancer ranks as the second leading cause of cancer-related mortality worldwide, necessitating surgical resection as the sole treatment option. Over the years, there has been a growing adoption of minimally invasive surgical techniques such as robotic and laparoscopic approaches. Robotic surgery represents an innovative modality that effectively addresses the limitations associated with traditional laparoscopic techniques. While previous studies have reported favorable perioperative outcomes for robot-assisted radical resection in rectal cancer patients, further evidence regarding its oncological safety is still warranted. AIM: To conduct a comparative analysis of perioperative and oncological outcomes between robot-assisted and laparoscopic-assisted low anterior resection (LALAR) procedures. METHODS: The clinical data of 125 patients who underwent robot-assisted low anterior resection (RALAR) and 279 patients who underwent LALAR resection at Shandong Provincial Hospital Affiliated to Shandong First Medical University from December 2019 to November 2022 were retrospectively analyzed. After performing a 1:1 propensity score matching, the patients were divided into two groups: The RALAR group and the LALAR group (111 cases in each group). Subsequently, a comparison was made between the short-term outcomes within 30 d after surgery and the 3-year survival outcomes of these two groups. RESULTS: Compared to the LALAR group, the RALAR group exhibited a significantly earlier time to first flatus [2 (2-2) d vs 3 (3-3) d, P = 0.000], as well as a shorter time to first fluid diet [4 (3-4) d vs 5 (4-6) d, P = 0.001]. Additionally, the RALAR group demonstrated reduced postoperative indwelling catheter time [2 (1-3) d vs 4 (3-5) d, P = 0.000] and decreased length of hospital stay after surgery [5 (5-7) d vs 7(6-8) d, P = 0.009]. Moreover, there was an observed increase in total cost of hospitalization for the RALAR group compared to the LALAR group [10777 (10780-11850) dollars vs 10550 (8766-11715) dollars, P = 0.012]. No significant differences were found in terms of conversion rate to laparotomy or incidence of postoperative complications between both groups. Furthermore, no significant disparities were noted regarding the 3-year overall survival rate and 3-year disease-free survival rate between both groups. CONCLUSION: Robotic surgery offers potential advantages in terms of accelerated recovery of gastrointestinal and urologic function compared to LALAR resection, while maintaining similar perioperative and 3-year oncological outcomes.
RESUMO
Background: Even though patients have easy access to their electronic health records and lab test results data through patient portals, lab results are often confusing and hard to understand. Many patients turn to online forums or question and answering (Q&A) sites to seek advice from their peers. However, the quality of answers from social Q&A on health-related questions varies significantly, and not all the responses are accurate or reliable. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to get their questions answered. Objective: We aim to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to lab test-related questions asked by patients and to identify potential issues that can be mitigated with augmentation approaches. Methods: We first collected lab test results related question and answer data from Yahoo! Answers and selected 53 Q&A pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from four LLMs including GPT-4, Meta LLaMA 2, MedAlpaca, and ORCA_mini. We first assessed the similarity of their answers using standard QA similarity-based evaluation metrics including ROUGE, BLEU, METEOR, BERTScore. We also utilized an LLM-based evaluator to judge whether a target model has higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. Finally, we performed a manual evaluation with medical experts for all the responses of seven selected questions on the same four aspects. Results: Regarding the similarity of the responses from 4 LLMs, where GPT-4 output was used as the reference answer, the responses from LLaMa 2 are the most similar ones, followed by LLaMa 2, ORCA_mini, and MedAlpaca. Human answers from Yahoo data were scored lowest and thus least similar to GPT-4-generated answers. The results of Win Rate and medical expert evaluation both showed that GPT-4's responses achieved better scores than all the other LLM responses and human responses on all the four aspects (relevance, correctness, helpfulness, and safety). However, LLM responses occasionally also suffer from lack of interpretation in one's medical context, incorrect statements, and lack of references. Conclusions: By evaluating LLMs in generating responses to patients' lab test results related questions, we find that compared to other three LLMs and human answer from the Q&A website, GPT-4's responses are more accurate, helpful, relevant, and safer. However, there are cases that GPT-4 responses are inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses including prompt engineering, prompt augmentation, retrieval augmented generation, and response evaluation.
RESUMO
To enable electronic screening of eligible patients for clinical trials, free-text clinical trial eligibility criteria should be translated to a computable format. Natural language processing (NLP) techniques have the potential to automate this process. In this study, we explored a supervised multi-input multi-output (MIMO) sequence labelling model to parse eligibility criteria into combinations of fact and condition tuples. Our experiments on a small manually annotated training dataset showed that that the performance of the MIMO framework with a BERT-based encoder using all the input sequences achieved an overall lenient-level AUROC of 0.61. Although the performance is suboptimal, representing eligibility criteria into logical and semantically clear tuples can potentially make subsequent translation of these tuples into database queries more reliable.
RESUMO
Viewing laboratory test results is patients' most frequent activity when accessing patient portals, but lab results can be very confusing for patients. Previous research has explored various ways to present lab results, but few have attempted to provide tailored information support based on individual patient's medical context. In this study, we collected and annotated interpretations of textual lab result in 251 health articles about laboratory tests from AHealthyMe.com. Then we evaluated transformer-based language models including BioBERT, ClinicalBERT, RoBERTa, and PubMedBERT for recognizing key terms and their types. Using BioPortal's term search API, we mapped the annotated terms to concepts in major controlled terminologies. Results showed that PubMedBERT achieved the best F1 on both strict and lenient matching criteria. SNOMED CT had the best coverage of the terms, followed by LOINC and ICD-10-CM. This work lays the foundation for enhancing the presentation of lab results in patient portals by providing patients with contextualized interpretations of their lab results and individualized question prompts that they can, in turn, refer to during physician consults.
Assuntos
Systematized Nomenclature of Medicine , Vocabulário Controlado , Humanos , Logical Observation Identifiers Names and Codes , Idioma , Armazenamento e Recuperação da InformaçãoRESUMO
To cope with the rapid growth of scientific publications and data in biomedical research, knowledge graphs (KGs) have emerged as a powerful data structure for integrating large volumes of heterogeneous data to facilitate accurate and efficient information retrieval and automated knowledge discovery (AKD). However, transforming unstructured content from scientific literature into KGs has remained a significant challenge, with previous methods unable to achieve human-level accuracy. In this study, we utilized an information extraction pipeline that won first place in the LitCoin NLP Challenge to construct a largescale KG using all PubMed abstracts. The quality of the large-scale information extraction rivals that of human expert annotations, signaling a new era of automatic, high-quality database construction from literature. Our extracted information markedly surpasses the amount of content in manually curated public databases. To enhance the KG's comprehensiveness, we integrated relation data from 40 public databases and relation information inferred from high-throughput genomics data. The comprehensive KG enabled rigorous performance evaluation of AKD, which was infeasible in previous studies. We designed an interpretable, probabilistic-based inference method to identify indirect causal relations and achieved unprecedented results for drug target identification and drug repurposing. Taking lung cancer as an example, we found that 40% of drug targets reported in literature could have been predicted by our algorithm about 15 years ago in a retrospective study, demonstrating that substantial acceleration in scientific discovery could be achieved through automated hypotheses generation and timely dissemination. A cloud-based platform (https://www.biokde.com) was developed for academic users to freely access this rich structured data and associated tools.