2024 GSRA Abstracts.

The American Academy of Pediatric Dentistry (AAPD) is dedicated to the advancement of research related to improving children's oral health and the specialty of pediatric dentistry. To fulfill part of this mission, AAPD supports many research awards, grants, and fellowships. The following research abstracts have been chosen by a subcommittee of the AAPD Council on Scientific Affairs.

Characterizing the Increase in Artificial Intelligence Content Detection in Oncology Scientific Abstracts From 2021 to 2023.

PURPOSE: Artificial intelligence (AI) models can generate scientific abstracts that are difficult to distinguish from the work of human authors. The use of AI in scientific writing and performance of AI detection tools are poorly characterized. METHODS: We extracted text from published scientific abstracts from the ASCO 2021-2023 Annual Meetings. Likelihood of AI content was evaluated by three detectors: GPTZero, Originality.ai, and Sapling. Optimal thresholds for AI content detection were selected using 100 abstracts from before 2020 as negative controls, and 100 produced by OpenAI's GPT-3 and GPT-4 models as positive controls. Logistic regression was used to evaluate the association of predicted AI content with submission year and abstract characteristics, and adjusted odds ratios (aORs) were computed. RESULTS: Fifteen thousand five hundred and fifty-three abstracts met inclusion criteria. Across detectors, abstracts submitted in 2023 were significantly more likely to contain AI content than those in 2021 (aOR range from 1.79 with Originality to 2.37 with Sapling). Online-only publication and lack of clinical trial number were consistently associated with AI content. With optimal thresholds, 99.5%, 96%, and 97% of GPT-3/4-generated abstracts were identified by GPTZero, Originality, and Sapling respectively, and no sampled abstracts from before 2020 were classified as AI generated by the GPTZero and Originality detectors. Correlation between detectors was low to moderate, with Spearman correlation coefficient ranging from 0.14 for Originality and Sapling to 0.47 for Sapling and GPTZero. CONCLUSION: There is an increasing signal of AI content in ASCO abstracts, coinciding with the growing popularity of generative AI models.

Geographic Barriers to Indexing: Examining Appalachia.

The objective of this study was to examine the accuracy of indexing for "Appalachian Region"[Mesh]. Researchers performed a search in PubMed for articles published in 2019 using "Appalachian Region"[Mesh] or "Appalachia" or "Appalachian" in the title or abstract. Only 17.88% of the articles retrieved by the search were about Appalachia according to the ARC definition. Most articles retrieved appeared because they were indexed with state terms that were included as part of the mesh term. Database indexing and searching transparency is of growing importance as indexers rely increasingly on automated systems to catalog information and publications.

Machine learning models for abstract screening task - A systematic literature review application for health economics and outcome research.

OBJECTIVE: Systematic literature reviews (SLRs) are critical for life-science research. However, the manual selection and retrieval of relevant publications can be a time-consuming process. This study aims to (1) develop two disease-specific annotated corpora, one for human papillomavirus (HPV) associated diseases and the other for pneumococcal-associated pediatric diseases (PAPD), and (2) optimize machine- and deep-learning models to facilitate automation of the SLR abstract screening. METHODS: This study constructed two disease-specific SLR screening corpora for HPV and PAPD, which contained citation metadata and corresponding abstracts. Performance was evaluated using precision, recall, accuracy, and F1-score of multiple combinations of machine- and deep-learning algorithms and features such as keywords and MeSH terms. RESULTS AND CONCLUSIONS: The HPV corpus contained 1697 entries, with 538 relevant and 1159 irrelevant articles. The PAPD corpus included 2865 entries, with 711 relevant and 2154 irrelevant articles. Adding additional features beyond title and abstract improved the performance (measured in Accuracy) of machine learning models by 3% for HPV corpus and 2% for PAPD corpus. Transformer-based deep learning models that consistently outperformed conventional machine learning algorithms, highlighting the strength of domain-specific pre-trained language models for SLR abstract screening. This study provides a foundation for the development of more intelligent SLR systems.

The reporting quality and spin of randomized controlled trials of endometriosis pain: Methodological study based on CONSORT extension on abstracts.

OBJECTIVE: To assess the reporting quality of published RCT abstracts regarding patients with endometriosis pelvic pain and investigate the prevalence and characteristics of spin in these abstracts. METHODS: PubMed and Scopus were searched for RCT abstracts addressing endometriosis pelvic pain published from January 1st, 2010 to December 1st, 2023.The reporting quality of RCT abstracts was assessed using the CONSORT statement for abstracts. Additionally, spin was evaluated in the results and conclusions section of the abstracts, defined as the misleading reporting of study findings to emphasize the perceived benefits of an intervention or to confound readers from statistically non-significant results. Assessing factors affecting the reporting quality and spin existence, linear and logistic regression was used, respectively. RESULTS: A total of 47 RCT abstracts were included. Out of 16 checklist items, only three items including objective, intervention and conclusions were sufficiently reported in the most abstracts (more than 95%), and none of the abstracts presented precise data as required by the CONSORT-A guidelines. In the reporting quality of material and method section, trial design, type of randomization, the generation of random allocation sequences, the allocation concealment and blinding were most items identified that were suboptimal. The total score for the quality varied between 5 and 15 (mean: 9.59, SD: 3.03, median: 9, IQR: 5). Word count (beta = 0.015, p-value = 0.005) and publishing in open-accessed journals (beta = 2.023, p-value = 0.023) were the significant factors that affecting the reporting quality. Evaluating spin within each included paper, we found that 18 (51.43%) papers had statistically non-significant results. From these studies, 12 (66.66%) had spin in both results and conclusion sections. Furthermore, the spin intensity increased during 2010-2023 and 38.29% of abstracts had spin in both results and conclusion sections. CONCLUSION: Overall poor adherence to CONSORT-A was observed, with spin detected in several RCTs featuring non-significant primary endpoints in obstetrics and gynecology literature.

Increasing Use of Promotional Language in Orthopaedic Surgery Abstracts-An Analysis of 112,916 Abstracts 1985 to 2020.

BACKGROUND: Increasing use of "hype" language (eg, language overstating research impact) has been documented in the scientific community. Evaluating language in abstracts is important because readers may use abstracts to extrapolate findings to entire publications. Our purpose was to assess the frequency of hype language within orthopaedic surgery. METHODS: One hundred thirty-nine hype adjectives were previously identified using a linguistics approach. All publicly available abstracts from 18 orthopaedic surgery journals between 1985 and 2020 were obtained, and hype adjectives were tabulated. Change in frequency of these adjectives was calculated. RESULTS: A total of 112,916 abstracts were identified. 67.0% (948/1414) of abstracts in 1985 contained hype adjectives, compared with 92.5% (5287/5714) in 2020. The average number of hype adjectives per abstract increased by 136% (1.1 to 2.6). Of the 139 adjectives, 87 (62.5%) increased in frequency and 40 (28.7%) decreased in frequency while 12 (9%) were not used. The hype adjectives with the largest absolute increases in frequency were quality (+324wpm), significant (+320wpm), systematic (+246wpm), top (+239wpm), and international (+201wpm). The five hype adjectives with the largest relative increases in frequency were novel (+10500%), international (+2850%), urgent (+2600%), robust (+2300%), and emerging (+1400%). CONCLUSION: Promotional language is increasing in orthopaedic surgery abstracts. Authors, editors, and reviewers should seek to minimize the usage of nonobjective language.

Fate of Pediatric Transport Medicine Abstracts 2011 to 2020: What Predicts Publication Success?

OBJECTIVE: Pediatric-neonatal transport research projects are presented at the American Academy of Pediatrics (AAP) Section on Transport Medicine (SOTM) scientific abstract program annually. Journal publication increases the impact of these projects. Our objectives were to determine the publication rate of transport abstracts and to identify factors predictive of publication success. METHODS: We reviewed all AAP SOTM abstracts accepted for presentation from 2011 to 2020 and assessed presentation format (oral/platform vs. poster), authors' professional degree (physician vs. nonphysician), and first author's trainee status. We searched PubMed, Ovid, and ResearchGate for publications by abstract title and authors and then compared published versus unpublished abstracts. Categorical variables were expressed as proportions and compared using the chi-square test or the Fisher exact test, whereas continuous variables were summarized using medians and interquartile ranges (IQRs) and compared using the Student t-test or the Kruskal-Wallis test as appropriate. A linear probability model was performed. RESULTS: Of 194 presented abstracts, 67 (34.5%) were published. The publication rate was significantly higher for oral/platform versus poster abstracts (P < .01), if the abstract was an oral/platform (probability increase by 19.5%, P < .01), and if the first author was a trainee (probability increase by 25.6%, p < 0.05). The constant was estimated as 24.9% probability of publication. Hence, if the first author was a physician, a trainee, and had an oral/platform presentation, there was an 85.8% chance of being published. The median (IQR) time to publication was 2 years (IQR: 2-4 years), with articles published the longest having the most citations. Articles were published in 27 different journals, with nearly half (33/67, 49.3%) being published in 3 journals. CONCLUSION: AAP SOTM abstracts have a 34.5% publication rate over the past 10 years, which is consistent with other medical specialties. Oral abstracts, physician first authors, and trainee first authors had a significantly higher success rate. Special emphasis should be placed nationally on supporting nonphysician transport professionals to publish their work.

Comparing generative and extractive approaches to information extraction from abstracts describing randomized clinical trials.

BACKGROUND: Systematic reviews of Randomized Controlled Trials (RCTs) are an important part of the evidence-based medicine paradigm. However, the creation of such systematic reviews by clinical experts is costly as well as time-consuming, and results can get quickly outdated after publication. Most RCTs are structured based on the Patient, Intervention, Comparison, Outcomes (PICO) framework and there exist many approaches which aim to extract PICO elements automatically. The automatic extraction of PICO information from RCTs has the potential to significantly speed up the creation process of systematic reviews and this way also benefit the field of evidence-based medicine. RESULTS: Previous work has addressed the extraction of PICO elements as the task of identifying relevant text spans or sentences, but without populating a structured representation of a trial. In contrast, in this work, we treat PICO elements as structured templates with slots to do justice to the complex nature of the information they represent. We present two different approaches to extract this structured information from the abstracts of RCTs. The first approach is an extractive approach based on our previous work that is extended to capture full document representations as well as by a clustering step to infer the number of instances of each template type. The second approach is a generative approach based on a seq2seq model that encodes the abstract describing the RCT and uses a decoder to infer a structured representation of a trial including its arms, treatments, endpoints and outcomes. Both approaches are evaluated with different base models on a manually annotated dataset consisting of RCT abstracts on an existing dataset comprising 211 annotated clinical trial abstracts for Type 2 Diabetes and Glaucoma. For both diseases, the extractive approach (with flan-t5-base) reached the best F 1 score, i.e. 0.547 ( ± 0.006 ) for type 2 diabetes and 0.636 ( ± 0.006 ) for glaucoma. Generally, the F 1 scores were higher for glaucoma than for type 2 diabetes and the standard deviation was higher for the generative approach. CONCLUSION: In our experiments, both approaches show promising performance extracting structured PICO information from RCTs, especially considering that most related work focuses on the far easier task of predicting less structured objects. In our experimental results, the extractive approach performs best in both cases, although the lead is greater for glaucoma than for type 2 diabetes. For future work, it remains to be investigated how the base model size affects the performance of both approaches in comparison. Although the extractive approach currently leaves more room for direct improvements, the generative approach might benefit from larger models.

The Impact of Visual Abstracts Compared to Automated Tweets on Social Media in Otology & Neurotology.

OBJECTIVE: To analyze the effect of visual abstracts versus automated tweets on social media participation in Otology & Neurotology . PATIENTS: N/A. INTERVENTIONS: Introduction of visual abstracts developed by the social media editorial team to established automated tweets created by the dlvr.it computer program on the Otology & Neurotology Twitter account. MAIN OUTCOME MEASURES: Twitter analytics including the number of new followers per month, impressions per tweet, and engagements per tweet. The Kruskal-Wallis analysis of variance test was used to compare means. RESULTS: From October 2016 to October 2017 (average of 20 new followers per month), 101 automated tweets averaged 536 impressions and 16 engagements per tweet. The visual abstract was introduced in November 2017. From November 2017 to November 2020 (average of 39 new followers per month), 447 automated tweets averaged 747 impressions and 22 engagements per tweet, whereas 157 visual abstracts averaged 1977 impressions and 78 engagements per tweet. Automated tweets were discontinued in December 2020. From December 2020 to December 2022 (average of 44 new followers per month), 95 visual abstracts averaged 1893 impressions and 103 engagements per tweet. With the introduction of the visual abstract, the average number of followers, impressions per tweet, and engagements per tweet significantly increased (all p -values <0.01; all large effect sizes of 0.16, 0.47, and 0.47, respectively). CONCLUSIONS: Visual abstracts created by a social media editorial team have a positive impact on social media participation in the field of otology and neurotology. The impact is greater than that of social media content generated by Twitter automation tools.

Association of reviewer experience with discriminating human-written versus ChatGPT-written abstracts.

OBJECTIVE: To determine if reviewer experience impacts the ability to discriminate between human-written and ChatGPT-written abstracts. METHODS: Thirty reviewers (10 seniors, 10 juniors, and 10 residents) were asked to differentiate between 10 ChatGPT-written and 10 human-written (fabricated) abstracts. For the study, 10 gynecologic oncology abstracts were fabricated by the authors. For each human-written abstract we generated a ChatGPT matching abstract by using the same title and the fabricated results of each of the human generated abstracts. A web-based questionnaire was used to gather demographic data and to record the reviewers' evaluation of the 20 abstracts. Comparative statistics and multivariable regression were used to identify factors associated with a higher correct identification rate. RESULTS: The 30 reviewers discriminated 20 abstracts, giving a total of 600 abstract evaluations. The reviewers were able to correctly identify 300/600 (50%) of the abstracts: 139/300 (46.3%) of the ChatGPT-generated abstracts and 161/300 (53.7%) of the human-written abstracts (p=0.07). Human-written abstracts had a higher rate of correct identification (median (IQR) 56.7% (49.2-64.1%) vs 45.0% (43.2-48.3%), p=0.023). Senior reviewers had a higher correct identification rate (60%) than junior reviewers and residents (45% each; p=0.043 and p=0.002, respectively). In a linear regression model including the experience level of the reviewers, familiarity with artificial intelligence (AI) and the country in which the majority of medical training was achieved (English speaking vs non-English speaking), the experience of the reviewer (ß=10.2 (95% CI 1.8 to 18.7)) and familiarity with AI (ß=7.78 (95% CI 0.6 to 15.0)) were independently associated with the correct identification rate (p=0.019 and p=0.035, respectively). In a correlation analysis the number of publications by the reviewer was positively correlated with the correct identification rate (r28)=0.61, p<0.001. CONCLUSION: A total of 46.3% of abstracts written by ChatGPT were detected by reviewers. The correct identification rate increased with reviewer and publication experience.