Analysis of the Types of Retracted COVID-19 Articles Published in PubMed-Listed Journals.

OBJECTIVES: Periodically, medical publications are retracted. The reasons vary from minor situations, such as author attributions, which do not undermine the validity of the data or the analysis in the article, to serious reasons, such as fraud. Understanding the reasons for retraction can provide important information for clinicians, educators, researchers, journals, and editorial boards. METHODS: The PubMed database was searched using the term "COVID-19" (coronavirus disease 2019) and the term limitation "retracted publication." The characteristics of the journals with retracted articles, the types of article, and the reasons for retraction were analyzed. RESULTS: This search recovered 196 articles that had been retracted. These retractions were published in 179 different journals; 14 journals had >1 retracted article. The mean impact factor of these journals was 8.4, with a range of 0.32-168.9. The most frequent reasons for retractions were duplicate publication, concerns about data validity and analysis, concerns about peer review, author request, and the lack of permission or ethical violation. There were significant differences between the types of article and the reasons for retraction but no consistent pattern. A more detailed analysis of two particular retractions demonstrates the complexity and the effort required to make decisions about article retractions. CONCLUSIONS: The retraction of published articles presents a significant challenge to journals, editorial boards, peer reviewers, and authors. This process has the potential to provide important benefits; it also has the potential to undermine confidence in both research and the editorial process.

TIN-X version 3: update with expanded dataset and modernized architecture for enhanced illumination of understudied targets.

TIN-X (Target Importance and Novelty eXplorer) is an interactive visualization tool for illuminating associations between diseases and potential drug targets and is publicly available at newdrugtargets.org. TIN-X uses natural language processing to identify disease and protein mentions within PubMed content using previously published tools for named entity recognition (NER) of gene/protein and disease names. Target data is obtained from the Target Central Resource Database (TCRD). Two important metrics, novelty and importance, are computed from this data and when plotted as log(importance) vs. log(novelty), aid the user in visually exploring the novelty of drug targets and their associated importance to diseases. TIN-X Version 3.0 has been significantly improved with an expanded dataset, modernized architecture including a REST API, and an improved user interface (UI). The dataset has been expanded to include not only PubMed publication titles and abstracts, but also full-text articles when available. This results in approximately 9-fold more target/disease associations compared to previous versions of TIN-X. Additionally, the TIN-X database containing this expanded dataset is now hosted in the cloud via Amazon RDS. Recent enhancements to the UI focuses on making it more intuitive for users to find diseases or drug targets of interest while providing a new, sortable table-view mode to accompany the existing plot-view mode. UI improvements also help the user browse the associated PubMed publications to explore and understand the basis of TIN-X's predicted association between a specific disease and a target of interest. While implementing these upgrades, computational resources are balanced between the webserver and the user's web browser to achieve adequate performance while accommodating the expanded dataset. Together, these advances aim to extend the duration that users can benefit from TIN-X while providing both an expanded dataset and new features that researchers can use to better illuminate understudied proteins.

GPT-4 performance on querying scientific publications: reproducibility, accuracy, and impact of an instruction sheet.

BACKGROUND: Large language models (LLMs) that can efficiently screen and identify studies meeting specific criteria would streamline literature reviews. Additionally, those capable of extracting data from publications would enhance knowledge discovery by reducing the burden on human reviewers. METHODS: We created an automated pipeline utilizing OpenAI GPT-4 32 K API version "2023-05-15" to evaluate the accuracy of the LLM GPT-4 responses to queries about published papers on HIV drug resistance (HIVDR) with and without an instruction sheet. The instruction sheet contained specialized knowledge designed to assist a person trying to answer questions about an HIVDR paper. We designed 60 questions pertaining to HIVDR and created markdown versions of 60 published HIVDR papers in PubMed. We presented the 60 papers to GPT-4 in four configurations: (1) all 60 questions simultaneously; (2) all 60 questions simultaneously with the instruction sheet; (3) each of the 60 questions individually; and (4) each of the 60 questions individually with the instruction sheet. RESULTS: GPT-4 achieved a mean accuracy of 86.9% - 24.0% higher than when the answers to papers were permuted. The overall recall and precision were 72.5% and 87.4%, respectively. The standard deviation of three replicates for the 60 questions ranged from 0 to 5.3% with a median of 1.2%. The instruction sheet did not significantly increase GPT-4's accuracy, recall, or precision. GPT-4 was more likely to provide false positive answers when the 60 questions were submitted individually compared to when they were submitted together. CONCLUSIONS: GPT-4 reproducibly answered 3600 questions about 60 papers on HIVDR with moderately high accuracy, recall, and precision. The instruction sheet's failure to improve these metrics suggests that more sophisticated approaches are necessary. Either enhanced prompt engineering or finetuning an open-source model could further improve an LLM's ability to answer questions about highly specialized HIVDR papers.

Validation of an interprofessional education search strategy in PubMed to optimize IPE literature searching.

Objective: With exponential growth in the publication of interprofessional education (IPE) research studies, it has become more difficult to find relevant literature and stay abreast of the latest research. To address this gap, we developed, evaluated, and validated search strategies for IPE studies in PubMed, to improve future access to and synthesis of IPE research. These search strategies, or search hedges, provide comprehensive, validated sets of search terms for IPE publications. Methods: The search strategies were created for PubMed using relative recall methodology. The research methods followed the guidance of previous search hedge and search filter validation studies in creating a gold standard set of relevant references using systematic reviews, having expert searchers identify and test search terms, and using relative recall calculations to validate the searches' performance against the gold standard set. Results: The three recommended search hedges for IPE studies presented had recall of 71.5%, 82.7%, and 95.1%; the first more focused for efficient literature searching, the last with high recall for comprehensive literature searching, and the remaining hedge as a middle ground between the other two options. Conclusion: These validated search hedges can be used in PubMed to expedite finding relevant scholarships, staying up to date with IPE research, and conducting literature reviews and evidence syntheses.

Meeting a need: development and validation of PubMed search filters for immigrant populations.

Objective: There is a need for additional comprehensive and validated filters to find relevant references more efficiently in the growing body of research on immigrant populations. Our goal was to create reliable search filters that direct librarians and researchers to pertinent studies indexed in PubMed about health topics specific to immigrant populations. Methods: We applied a systematic and multi-step process that combined information from expert input, authoritative sources, automation, and manual review of sources. We established a focused scope and eligibility criteria, which we used to create the development and validation sets. We formed a term ranking system that resulted in the creation of two filters: an immigrant-specific and an immigrant-sensitive search filter. Results: When tested against the validation set, the specific filter sensitivity was 88.09%, specificity 97.26%, precision 97.88%, and the NNR 1.02. The sensitive filter sensitivity was 97.76%when tested against the development set. The sensitive filter had a sensitivity of 97.14%, specificity of 82.05%, precision of 88.59%, accuracy of 90.94%, and NNR [See Table 1] of 1.13 when tested against the validation set. Conclusion: We accomplished our goal of developing PubMed search filters to help researchers retrieve studies about immigrants. The specific and sensitive PubMed search filters give information professionals and researchers options to maximize the specificity and precision or increase the sensitivity of their search for relevant studies in PubMed. Both search filters generated strong performance measurements and can be used as-is, to capture a subset of immigrant-related literature, or adapted and revised to fit the unique research needs of specific project teams (e.g. remove US-centric language, add location-specific terminology, or expand the search strategy to include terms for the topic/s being investigated in the immigrant population identified by the filter). There is also a potential for teams to employ the search filter development process described here for their own topics and use.

Enhancing the coverage of SemRep using a relation classification approach.

OBJECTIVE: Relation extraction is an essential task in the field of biomedical literature mining and offers significant benefits for various downstream applications, including database curation, drug repurposing, and literature-based discovery. The broad-coverage natural language processing (NLP) tool SemRep has established a solid baseline for extracting subject-predicate-object triples from biomedical text and has served as the backbone of the Semantic MEDLINE Database (SemMedDB), a PubMed-scale repository of semantic triples. While SemRep achieves reasonable precision (0.69), its recall is relatively low (0.42). In this study, we aimed to enhance SemRep using a relation classification approach, in order to eventually increase the size and the utility of SemMedDB. METHODS: We combined and extended existing SemRep evaluation datasets to generate training data. We leveraged the pre-trained PubMedBERT model, enhancing it through additional contrastive pre-training and fine-tuning. We experimented with three entity representations: mentions, semantic types, and semantic groups. We evaluated the model performance on a portion of the SemRep Gold Standard dataset and compared it to SemRep performance. We also assessed the effect of the model on a larger set of 12K randomly selected PubMed abstracts. RESULTS: Our results show that the best model yields a precision of 0.62, recall of 0.81, and F1 score of 0.70. Assessment on 12K abstracts shows that the model could double the size of SemMedDB, when applied to entire PubMed. We also manually assessed the quality of 506 triples predicted by the model that SemRep had not previously identified, and found that 67% of these triples were correct. CONCLUSION: These findings underscore the promise of our model in achieving a more comprehensive coverage of relationships mentioned in biomedical literature, thereby showing its potential in enhancing various downstream applications of biomedical literature mining. Data and code related to this study are available at https://github.com/Michelle-Mings/SemRep_RelationClassification.

Geographic Barriers to Indexing: Examining Appalachia.

The objective of this study was to examine the accuracy of indexing for "Appalachian Region"[Mesh]. Researchers performed a search in PubMed for articles published in 2019 using "Appalachian Region"[Mesh] or "Appalachia" or "Appalachian" in the title or abstract. Only 17.88% of the articles retrieved by the search were about Appalachia according to the ARC definition. Most articles retrieved appeared because they were indexed with state terms that were included as part of the mesh term. Database indexing and searching transparency is of growing importance as indexers rely increasingly on automated systems to catalog information and publications.

Utilizing ChatGPT as a scientific reasoning engine to differentiate conflicting evidence and summarize challenges in controversial clinical questions.

OBJECTIVE: Synthesizing and evaluating inconsistent medical evidence is essential in evidence-based medicine. This study aimed to employ ChatGPT as a sophisticated scientific reasoning engine to identify conflicting clinical evidence and summarize unresolved questions to inform further research. MATERIALS AND METHODS: We evaluated ChatGPT's effectiveness in identifying conflicting evidence and investigated its principles of logical reasoning. An automated framework was developed to generate a PubMed dataset focused on controversial clinical topics. ChatGPT analyzed this dataset to identify consensus and controversy, and to formulate unsolved research questions. Expert evaluations were conducted 1) on the consensus and controversy for factual consistency, comprehensiveness, and potential harm and, 2) on the research questions for relevance, innovation, clarity, and specificity. RESULTS: The gpt-4-1106-preview model achieved a 90% recall rate in detecting inconsistent claim pairs within a ternary assertions setup. Notably, without explicit reasoning prompts, ChatGPT provided sound reasoning for the assertions between claims and hypotheses, based on an analysis grounded in relevance, specificity, and certainty. ChatGPT's conclusions of consensus and controversies in clinical literature were comprehensive and factually consistent. The research questions proposed by ChatGPT received high expert ratings. DISCUSSION: Our experiment implies that, in evaluating the relationship between evidence and claims, ChatGPT considered more detailed information beyond a straightforward assessment of sentimental orientation. This ability to process intricate information and conduct scientific reasoning regarding sentiment is noteworthy, particularly as this pattern emerged without explicit guidance or directives in prompts, highlighting ChatGPT's inherent logical reasoning capabilities. CONCLUSION: This study demonstrated ChatGPT's capacity to evaluate and interpret scientific claims. Such proficiency can be generalized to broader clinical research literature. ChatGPT effectively aids in facilitating clinical studies by proposing unresolved challenges based on analysis of existing studies. However, caution is advised as ChatGPT's outputs are inferences drawn from the input literature and could be harmful to clinical practice.

PubTator 3.0: an AI-powered literature resource for unlocking biomedical knowledge.

PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text articles from the PMC open access subset, updated weekly. PubTator 3.0's online interface and API utilize these precomputed entity relations and synonyms to provide advanced search capabilities and enable large-scale analyses, streamlining many complex information needs. We showcase the retrieval quality of PubTator 3.0 using a series of entity pair queries, demonstrating that PubTator 3.0 retrieves a greater number of articles than either PubMed or Google Scholar, with higher precision in the top 20 results. We further show that integrating ChatGPT (GPT-4) with PubTator APIs dramatically improves the factuality and verifiability of its responses. In summary, PubTator 3.0 offers a comprehensive set of features and tools that allow researchers to navigate the ever-expanding wealth of biomedical literature, expediting research and unlocking valuable insights for scientific discovery.

CoMentG: comprehensive retrieval of generic relationships between biomedical concepts from the scientific literature.

The CoMentG resource contains millions of relationships between terms of biomedical interest obtained from the scientific literature. At the core of the system is a methodology for detecting significant co-mentions of concepts in the entire PubMed corpus. That method was applied to nine sets of terms covering the most important classes of biomedical concepts: diseases, symptoms/clinical signs, molecular functions, biological processes, cellular compartments, anatomic parts, cell types, bacteria and chemical compounds. We obtained more than 7 million relationships between more than 74 000 terms, and many types of relationships were not available in any other resource. As the terms were obtained from widely used resources and ontologies, the relationships are given using the standard identifiers provided by them and hence can be linked to other data. A web interface allows users to browse these associations, searching for relationships for a set of terms of interests provided as input, such as between a disease and their associated symptoms, underlying molecular processes or affected tissues. The results are presented in an interactive interface where the user can explore the reported relationships in different ways and follow links to other resources. Database URL: https://csbg.cnb.csic.es/CoMentG/.

Assessment of transparency indicators in space medicine.

Space medicine is a vital discipline with often time-intensive and costly projects and constrained opportunities for studying various elements such as space missions, astronauts, and simulated environments. Moreover, private interests gain increasing influence in this discipline. In scientific disciplines with these features, transparent and rigorous methods are essential. Here, we undertook an evaluation of transparency indicators in publications within the field of space medicine. A meta-epidemiological assessment of PubMed Central Open Access (PMC OA) eligible articles within the field of space medicine was performed for prevalence of code sharing, data sharing, pre-registration, conflicts of interest, and funding. Text mining was performed with the rtransparent text mining algorithms with manual validation of 200 random articles to obtain corrected estimates. Across 1215 included articles, 39 (3%) shared code, 258 (21%) shared data, 10 (1%) were registered, 110 (90%) contained a conflict-of-interest statement, and 1141 (93%) included a funding statement. After manual validation, the corrected estimates for code sharing, data sharing, and registration were 5%, 27%, and 1%, respectively. Data sharing was 32% when limited to original articles and highest in space/parabolic flights (46%). Overall, across space medicine we observed modest rates of data sharing, rare sharing of code and almost non-existent protocol registration. Enhancing transparency in space medicine research is imperative for safeguarding its scientific rigor and reproducibility.

Salivary and serum levels of lactate dehydrogenase in oral submucous fibrosis: A meta-analysis.

BACKGROUND: The occurrence of oral submucous fibrosis (OSF) is often accompanied by an increase in lactate dehydrogenase (LDH) levels. In this meta-analysis, we compared the salivary and serum levels of LDH levels between OSF patients and controls. MATERIAL AND METHODS: A comprehensive search was conducted in PubMed, Embase, Web of Science, and Cochrane Library from the establishment of the database to June 2023, and the quality of the studies was checked by the Newcastle-Ottawa Quality Assessment scale. The mean difference (MD) and 95% confidence interval (CI) were calculated using RevMan 5.4 software. RESULTS: A total of 28 studies were retrieved from the database, and we included 5 studies in this meta-analysis. The salivary LDH level of OSF patients was higher than healthy controls (MD: 423.10 pg/L 95%CI: 276.42-569.77 pg/mL, P < .00001), the serum LDH level of OSF patients was also higher than that of healthy controls (MD: 226.20 pg/mL, 95%CI: 147.71-304.69 pg/mL, P < .00001). CONCLUSIONS: This meta-analysis showed that salivary and serum LDH levels were higher in OSF patients than in healthy controls, suggesting that LDH may be a potential biomarker for OSF.

Deciphering the intricate linkage between the gut microbiota and Alzheimer's disease: Elucidating mechanistic pathways promising therapeutic strategies.

BACKGROUND: The gut microbiome is composed of various microorganisms such as bacteria, fungi, and protozoa, and constitutes an important part of the human gut. Its composition is closely related to human health and disease. Alzheimer's disease (AD) is a neurodegenerative disease whose underlying mechanism has not been fully elucidated. Recent research has shown that there are significant differences in the gut microbiota between AD patients and healthy individuals. Changes in the composition of gut microbiota may lead to the development of harmful factors associated with AD. In addition, the gut microbiota may play a role in the development and progression of AD through the gut-brain axis. However, the exact nature of this relationship has not been fully understood. AIMS: This review will elucidate the types and functions of gut microbiota and their relationship with AD and explore in depth the potential mechanisms of gut microbiota in the occurrence of AD and the prospects for treatment strategies. METHODS: Reviewed literature from PubMed and Web of Science using key terminologies related to AD and the gut microbiome. RESULTS: Research indicates that the gut microbiota can directly or indirectly influence the occurrence and progression of AD through metabolites, endotoxins, and the vagus nerve. DISCUSSION: This review discusses the future challenges and research directions regarding the gut microbiota in AD. CONCLUSION: While many unresolved issues remain regarding the gut microbiota and AD, the feasibility and immense potential of treating AD by modulating the gut microbiota are evident.

NetMe 2.0: a web-based platform for extracting and modeling knowledge from biomedical literature as a labeled graph.

MOTIVATION: The rapid increase of bio-medical literature makes it harder and harder for scientists to keep pace with the discoveries on which they build their studies. Therefore, computational tools have become more widespread, among which network analysis plays a crucial role in several life-science contexts. Nevertheless, building correct and complete networks about some user-defined biomedical topics on top of the available literature is still challenging. RESULTS: We introduce NetMe 2.0, a web-based platform that automatically extracts relevant biomedical entities and their relations from a set of input texts-i.e. in the form of full-text or abstract of PubMed Central's papers, free texts, or PDFs uploaded by users-and models them as a BioMedical Knowledge Graph (BKG). NetMe 2.0 also implements an innovative Retrieval Augmented Generation module (Graph-RAG) that works on top of the relationships modeled by the BKG and allows the distilling of well-formed sentences that explain their content. The experimental results show that NetMe 2.0 can infer comprehensive and reliable biological networks with significant Precision-Recall metrics when compared to state-of-the-art approaches. AVAILABILITY AND IMPLEMENTATION: https://netme.click/.

Text mining and portal development for gene-specific publications on Alzheimer's disease and other neurodegenerative diseases.

BACKGROUND: Tremendous research efforts have been made in the Alzheimer's disease (AD) field to understand the disease etiology, progression and discover treatments for AD. Many mechanistic hypotheses, therapeutic targets and treatment strategies have been proposed in the last few decades. Reviewing previous work and staying current on this ever-growing body of AD publications is an essential yet difficult task for AD researchers. METHODS: In this study, we designed and implemented a natural language processing (NLP) pipeline to extract gene-specific neurodegenerative disease (ND) -focused information from the PubMed database. The collected publication information was filtered and cleaned to construct AD-related gene-specific publication profiles. Six categories of AD-related information are extracted from the processed publication data: publication trend by year, dementia type occurrence, brain region occurrence, mouse model information, keywords occurrence, and co-occurring genes. A user-friendly web portal is then developed using Django framework to provide gene query functions and data visualizations for the generalized and summarized publication information. RESULTS: By implementing the NLP pipeline, we extracted gene-specific ND-related publication information from the abstracts of the publications in the PubMed database. The results are summarized and visualized through an interactive web query portal. Multiple visualization windows display the ND publication trends, mouse models used, dementia types, involved brain regions, keywords to major AD-related biological processes, and co-occurring genes. Direct links to PubMed sites are provided for all recorded publications on the query result page of the web portal. CONCLUSION: The resulting portal is a valuable tool and data source for quick querying and displaying AD publications tailored to users' interested research areas and gene targets, which is especially convenient for users without informatic mining skills. Our study will not only keep AD field researchers updated with the progress of AD research, assist them in conducting preliminary examinations efficiently, but also offers additional support for hypothesis generation and validation which will contribute significantly to the communication, dissemination, and progress of AD research.

Utilization of EHRs for clinical trials: a systematic review.

BACKGROUND AND OBJECTIVE: Clinical trials are of high importance for medical progress. This study conducted a systematic review to identify the applications of EHRs in supporting and enhancing clinical trials. MATERIALS AND METHODS: A systematic search of PubMed was conducted on 12/3/2023 to identify relevant studies on the use of EHRs in clinical trials. Studies were included if they (1) were full-text journal articles, (2) were written in English, (3) examined applications of EHR data to support clinical trial processes (e.g. recruitment, screening, data collection). A standardized form was used by two reviewers to extract data on: study design, EHR-enabled process(es), related outcomes, and limitations. RESULTS: Following full-text review, 19 studies met the predefined eligibility criteria and were included. Overall, included studies consistently demonstrated that EHR data integration improves clinical trial feasibility and efficiency in recruitment, screening, data collection, and trial design. CONCLUSIONS: According to the results of the present study, the use of Electronic Health Records in conducting clinical trials is very helpful. Therefore, it is better for researchers to use EHR in their studies for easy access to more accurate and comprehensive data. EHRs collects all individual data, including demographic, clinical, diagnostic, and therapeutic data. Moreover, all data is available seamlessly in EHR. In future studies, it is better to consider the cost-effectiveness of using EHR in clinical trials.

Raw data were not disclosed in 95% of PubMed-indexed heart failure meta-analyses in 2021: A systematic analysis of transparency.

BACKGROUND: The rising concern of irreproducible and non-transparent studies poses a significant challenge in modern medical literature. The impact of this issue on cardiology, particularly in the subfield of heart failure, remains poorly understood. To address this knowledge gap, we assessed the quality of evidence presented in recent heart failure meta-analyses by exploring several crucial transparency indicators. METHODS: We conducted a cross-sectional study and searched PubMed for meta - analyses themed around heart failure. We included the 100 most recent publications from 2021 and investigated the presence of several indices that are associated with transparency and reproducibility. RESULTS: The vast majority of the papers did not include their raw data (95/100, 95%) nor their analytic code (99/100, 99%). Less than half (42/100, 42%) preregistered their protocol, while only 65/100 (65%) adhered to a reporting guidelines method. Bias calculation for the respective studies included in each meta - analysis was present in 83/100 (83%) papers and publication bias was measured in approximately half (56/100, 56%). CONCLUSIONS: Our study indicates that meta-analyses in the field of heart failure present important information of transparency infrequently. Therefore, reproduction and validation of their findings seems to be practically impossible.

MetaTron: advancing biomedical annotation empowering relation annotation and collaboration.

BACKGROUND: The constant growth of biomedical data is accompanied by the need for new methodologies to effectively and efficiently extract machine-readable knowledge for training and testing purposes. A crucial aspect in this regard is creating large, often manually or semi-manually, annotated corpora vital for developing effective and efficient methods for tasks like relation extraction, topic recognition, and entity linking. However, manual annotation is expensive and time-consuming especially if not assisted by interactive, intuitive, and collaborative computer-aided tools. To support healthcare experts in the annotation process and foster annotated corpora creation, we present MetaTron. MetaTron is an open-source and free-to-use web-based annotation tool to annotate biomedical data interactively and collaboratively; it supports both mention-level and document-level annotations also integrating automatic built-in predictions. Moreover, MetaTron enables relation annotation with the support of ontologies, functionalities often overlooked by off-the-shelf annotation tools. RESULTS: We conducted a qualitative analysis to compare MetaTron with a set of manual annotation tools including TeamTat, INCEpTION, LightTag, MedTAG, and brat, on three sets of criteria: technical, data, and functional. A quantitative evaluation allowed us to assess MetaTron performances in terms of time and number of clicks to annotate a set of documents. The results indicated that MetaTron fulfills almost all the selected criteria and achieves the best performances. CONCLUSIONS: MetaTron stands out as one of the few annotation tools targeting the biomedical domain supporting the annotation of relations, and fully customizable with documents in several formats-PDF included, as well as abstracts retrieved from PubMed, Semantic Scholar, and OpenAIRE. To meet any user need, we released MetaTron both as an online instance and as a Docker image locally deployable.