Deep learning to refine the identification of high-quality clinical research articles from the biomedical literature: Performance evaluation.

BACKGROUND: Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence. OBJECTIVE: To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice. METHODS: We fine-tuned variations of BERT models (BERTBASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012 to 2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset. RESULTS: In training, three of the four selected best performing models were trained using BioBERTBASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles. CONCLUSIONS: Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.

A Deep Learning Approach to Refine the Identification of High-Quality Clinical Research Articles From the Biomedical Literature: Protocol for Algorithm Development and Validation.

BACKGROUND: A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence. OBJECTIVE: The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature. METHODS: Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores. RESULTS: Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022. CONCLUSIONS: The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/29398.

Role of solution chemistry in the retention and release of graphene oxide nanomaterials in uncoated and iron oxide-coated sand.

Understanding the fate and transport including remobilization of graphene oxide nanomaterials (GONMs) in the subsurface would enable us to expedite their benign use and evaluate their environmental impacts and health risks. In this study, the retention and release of GONMs were investigated in water-saturated columns packed with uncoated sand (Un-S) or iron oxide-coated sand (FeS) at environmentally relevant solution chemistries (1-100mM KCl and 0.1-10mM CaCl2 at pH7 and 11). Our results showed that increasing ionic strength (IS) inhibited GONMs' transport, and the impact of K+ was less than Ca2+. The positively charged iron oxide coating on sand surfaces immobilized the negatively charged GONMs (pH7) in the primary minimum, yielding hyperexponential retention profiles particularly in Ca2+. A stepwise decrease in pore-water IS caused detachment of previously retained GONMs. The mass of GONMs released during each detachment step correlated positively with the difference in secondary minimum depth (ΔΦmin2) at each IS, indicating that the released GONMs were retained in the secondary minimum. While most retained GONMs were re-entrained upon lowering pore-water IS in Un-S, decreasing IS only released limited GONMs in FeS, which were captured in the primary minimum. Introducing 1mM NaOH (pH11) released most retained GONMs in FeS; and average hydrodynamic diameters of the detached GONMs upon injecting NaOH were significantly smaller than those of GONMs in the influent and retentate, suggesting that NaOH induced GONMs disaggregation. Our findings advance current knowledge to better predict NMs' fate and transport under various solution chemistries such as during rainfall events or in the mixing zones between sea water and fresh water where transient IS changes drastically.

Transport behavior of humic acid-modified nano-hydroxyapatite in saturated packed column: effects of Cu, ionic strength, and ionic composition.

The surfaces of nano-hydroxyapatite (nHAP) used for contaminated soil and groundwater remediation may be modified to render nHAP highly mobile in the subsurface. Humic acid (HA) is widely used to modify and stabilize colloid suspensions. In this work, column experiments were conducted to determine the effects of contaminant (e.g., Cu) concentration, ionic strength (IS), and ion composition (IC) on the transport behavior of HA-modified nHAP in saturated packed columns. IS and nature of the cation had strong effects on the deposition of nHAP, and the effect was greater for divalent than for monovalent cations. Divalent cations have a greater capacity to screen the surface charge of nHAP, and Ca(2+) bridges the HA-modified nHAP colloidal particles, which causes greater deposition. Moreover, Cu(2+) had a greater effect on the transport behavior than Ca(2+) due to their strong exchange with Ca(2+) of nHAP and its surface complexation with nHAP. The relative travel distance L(T), of the injected HA-modified nHAP colloids, ranges from less than one to several meters at varying Cu concentrations, ISs, and ICs in saturated packed columns. The results are crucial to evaluate the efficacy of nHAP on the remediation of contaminated soil and groundwater environments.

[Transport of hydroxyapatite nanoparticles in saturated packed column: effects of humic acid, pH and ionic strengths].

Quartz sand was selected as collector and saturated packed column was constructed to explore the effects of environmental factors (humic acid, pH and ionic strengths of the bulk solution) on the transport and fate of hydroxyapatite nanoparticles (Nano-HAP) through measuring zeta potentials and representative c(i)/c(0) of Nano-HAP. It was suggested that zeta potentials of Nano-HAP colloids became more negative with increasing humic acid concentration and the change in solution composition from 0 to 10 mg/L humic acid yielded an increase in the zeta potentials of Nano-HAP colloids from -15 mV to -55 mV and a sharp decrease in a (attachment efficiency) from 1.0 to 0.012, meanwhile, the increase in bulk solution pH yielded a slight decrease in a which enhancing its transportation in saturated packed column. However, zeta potentials of Nano-HAP colloids became less negative as the ionic strength of bulk solution increased due to the compression of diffuse double layer and yielded an increase in a which greatly impeded its mobility during the pore-water solution, meanwhile, divalent cations have significantly stronger influence on the transport of Nano-HAP than monovalent cations of the bulk solution. The increase in the concentration of monovalent cation (Na+) from 1 to 100 mmol/L yielded an increase in a from 0.030 to 0.13, and divalent cations (Ca2+) from 0.2 to 10 mmol/L yielded a greatly increase in alpha from 0.030 to 1.0. It is important to note that the results could considerably contribute to gain insights in the transport and fate of Nano-HAP in natured and engineered porous media.

Facilitated transport of Cu with hydroxyapatite nanoparticles in saturated sand: effects of solution ionic strength and composition.

Column experiments were conducted to investigate the facilitated transport of Cu in association with hydroxyapatite nanoparticles (nHAP) in water-saturated quartz sand at different solution concentrations of NaCl (0-100 mM) or CaCl(2) (0.1-1.0 mM). The experimental breakthrough curves and retention profiles of nHAP were well described using a mathematical model that accounted for two kinetic retention sites. The retention coefficients for both sites increased with the ionic strength (IS) of a particular salt. However, the amount of nHAP retention was more sensitive to increases in the concentration of divalent Ca(2+) than monovalent Na(+). The effluent concentration of Cu that was associated with nHAP decreased significantly from 2.62 to 0.17 mg L(-1) when NaCl increased from 0 to 100 mM, and from 1.58 to 0.16 mg L(-1) when CaCl(2) increased from 0.1 to 1.0 mM. These trends were due to enhanced retention of nHAP with changes in IS and ionic composition (IC) due to compression of the double layer thickness and reduction of the magnitude of the zeta potentials. Results indicate that the IS and IC had a strong influence on the co-transport behavior of contaminants with nHAP nanoparticles.