Natural Language Processing for Literature Search in Vascular Surgery: A Pilot Study Testing an Artificial Intelligence Based Application.

ABSTRACT

Introduction: The use of natural language processing (NLP) for a literature search has been poorly investigated in vascular surgery so far. The aim of this pilot study was to test the applicability of an artificial intelligence (AI) based mobile application for literature searching in a topic related to vascular surgery. Technique A focused scientific question was defined to evaluate the performance of the AI application for a literature search and compare the results with the ground truth provided via a traditional literature search performed by human experts. Using pre-defined keywords, the literature search was performed automatically by the AI application through different steps, including quality assessment based on evaluation of the information available and quality filters using indicators of level of evidence, selection of publications based on relevancy filters using NLP, summarisation, and visualisation of the publications via the mobile app. A traditional literature search performed by human experts required 10 hours to check 154 original articles, among which 26 (16.9%) were truly related to the question, 63 (40.9%) related to the field but not to the specific question, and 65 (42.2%) were unrelated. The AI based search was performed in less than one hour, and, compared with traditional search, the method identified 17 original articles (48.6%) truly related to the question (p < .010), 18 (51.4%) related to the field but not to the specific question (p = .26), and no unrelated publications (p < .001). Fifteen truly related articles (88.2%) were identified jointly by the two methods. No significant difference was observed regarding the median number of citations, year of publications, and impact factor of journals. Discussion: The AI based method enabled a targeted, focused, and time saving literature search, although the selection of publications was not completely exhaustive. These results suggest that such an AI driven application is a complementary tool to help researchers and clinicians for continuous education and dissemination of knowledge.