Harnessing artificial intelligence in bariatric surgery: comparative analysis of ChatGPT-4, Bing, and Bard in generating clinician-level bariatric surgery recommendations.

BACKGROUND: The formulation of clinical recommendations pertaining to bariatric surgery is essential in guiding healthcare professionals. However, the extensive and continuously evolving body of literature in bariatric surgery presents considerable challenge for staying abreast of latest developments and efficient information acquisition. Artificial intelligence (AI) has the potential to streamline access to the salient points of clinical recommendations in bariatric surgery. OBJECTIVES: The study aims to appraise the quality and readability of AI-chat-generated answers to frequently asked clinical inquiries in the field of bariatric and metabolic surgery. SETTING: Remote. METHODS: Question prompts inputted into AI large language models (LLMs) and were created based on pre-existing clinical practice guidelines regarding bariatric and metabolic surgery. The prompts were queried into 3 LLMs: OpenAI ChatGPT-4, Microsoft Bing, and Google Bard. The responses from each LLM were entered into a spreadsheet for randomized and blinded duplicate review. Accredited bariatric surgeons in North America independently assessed appropriateness of each recommendation using a 5-point Likert scale. Scores of 4 and 5 were deemed appropriate, while scores of 1-3 indicated lack of appropriateness. A Flesch Reading Ease (FRE) score was calculated to assess the readability of responses generated by each LLMs. RESULTS: There was a significant difference between the 3 LLMs in their 5-point Likert scores, with mean values of 4.46 (SD .82), 3.89 (.80), and 3.11 (.72) for ChatGPT-4, Bard, and Bing (P < .001). There was a significant difference between the 3 LLMs in the proportion of appropriate answers, with ChatGPT-4 at 85.7%, Bard at 74.3%, and Bing at 25.7% (P < .001). The mean FRE scores for ChatGPT-4, Bard, and Bing, were 21.68 (SD 2.78), 42.89 (4.03), and 14.64 (5.09), respectively, with higher scores representing easier readability. CONCLUSIONS: LLM-based AI chat models can effectively generate appropriate responses to clinical questions related to bariatric surgery, though the performance of different models can vary greatly. Therefore, caution should be taken when interpreting clinical information provided by LLMs, and clinician oversight is necessary to ensure accuracy. Future investigation is warranted to explore how LLMs might enhance healthcare provision and clinical decision-making in bariatric surgery.

YouTube as a Source of Patient and Trainee Education in Vascular Surgery: A Systematic Review.

Objective: Due to its video based approach, YouTube has become a widely accessed educational resource for patients and trainees. This systematic review characterised and evaluated the peer reviewed literature investigating YouTube as a source of patient or trainee education in vascular surgery. Data sources: A comprehensive literature search was conducted using EMBASE, MEDLINE, and Ovid HealthStar from inception until 19 January 2023. All primary studies and conference abstracts evaluating YouTube as a source of vascular surgery education were included. Review methods: Video educational quality was analysed across several factors, including pathology, video audience, and length. Results: Overall, 24 studies were identified examining 3 221 videos with 123.1 hours of content and 37.1 million views. Studies primarily examined YouTube videos on diabetic foot care (7/24, 29%), peripheral arterial disease (3/24, 13%), carotid artery stenosis (3/24, 13%), varicose veins (3/24, 13%), and abdominal aortic aneurysm (2/24, 8%). Video educational quality was analysed using standardised assessment tools, author generated scoring systems, or global author reported assessment of quality. Six studies assessed videos for trainee education, while 18 studies evaluated videos for patient education. Among the 20 studies which reported on the overall quality of educational content, 10/20 studies deemed it poor, and 10/20 studies considered it fair, with 53% of studies noting poor educational quality for videos intended for patients and 40% of studies noting poor educational quality in videos intended for trainees. Poor quality videos had more views than fair quality videos (mean 27 348, 95% CI 15 154-39 543 views vs. 11 372, 95% CI 3 115-19 629 views, p = .030). Conclusion: The overall educational quality of YouTube videos for vascular surgery patient and trainee education is suboptimal. There is significant heterogeneity in the quality assessment tools used in their evaluation. A standardised approach to online education with a consistent quality assessment tool is required to better support online patient and trainee education in vascular surgery.

Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index.

OBJECTIVE: The fragility index (FI) measures the robustness of statistically significant findings in randomized controlled trials (RCTs) by quantifying the minimum number of event conversions required to reverse a dichotomous outcome's statistical significance. In vascular surgery, many clinical guidelines and critical decision-making points are informed by a handful of key RCTs, especially regarding open surgical versus endovascular treatment. The objective of this study is to evaluate the FI of RCTs with statistically significant primary outcomes that compared open vs endovascular surgery in vascular surgery. METHODS: In this meta-epidemiological study and systematic review, MEDLINE, Embase, and CENTRAL were searched for RCTs comparing open versus endovascular treatments for abdominal aortic aneurysms, carotid artery stenosis, and peripheral arterial disease to December 2022. RCTs with statistically significant primary outcomes were included. Data screening and extraction were conducted in duplicate. The FI was calculated by adding an event to the group with the smaller number of events while subtracting a nonevent to the same group until Fisher's exact test produced a nonstatistically significant result. The primary outcome was the FI and proportion of outcomes where the loss to follow-up was greater than the FI. The secondary outcomes assessed the relationship of the FI to disease state, presence of commercial funding, and study design. RESULTS: Overall, 5133 articles were captured in the initial search with 21 RCTs reporting 23 different primary outcomes being included in the final analysis. The median FI (first quartile, third quartile) was 3 (3, 20) with 16 outcomes (70%) reporting a loss to follow-up greater than its FI. Mann-Whitney U test revealed that commercially funded RCTs and composite outcomes had greater FIs (median, 20.0 [5.5, 24.5] vs median, 3.0 [2.0, 5.5], P = .035; median, 21 [8, 38] vs median, 3.0 [2.0, 8.5], P = .01, respectively). The FI did not vary between disease states (P = .285) or between index and follow-up trials (P = .147). There were significant correlations between the FI and P values (Pearson r = 0.90; 95% confidence interval, 0.77-0.96), and the number of events (r = 0.82; 95% confidence interval, 0.48-0.97). CONCLUSIONS: A small number of event conversions (median, 3) are needed to alter the statistical significance of primary outcomes in vascular surgery RCTs evaluating open surgical and endovascular treatments. Most studies had loss to follow-up greater than its FI, which can call into question trial results, and commercially funded studies had a greater FI. The FI and these findings should be considered in future trial design in vascular surgery.

Development of an Artificial Intelligence Tool for Intraoperative Guidance During Endovascular Abdominal Aortic Aneurysm Repair.

BACKGROUND: Adverse events during surgery can occur in part due to errors in visual perception and judgment. Deep learning is a branch of artificial intelligence (AI) that has shown promise in providing real-time intraoperative guidance. This study aims to train and test the performance of a deep learning model that can identify inappropriate landing zones during endovascular aneurysm repair (EVAR). METHODS: A deep learning model was trained to identify a "No-Go" landing zone during EVAR, defined by coverage of the lowest renal artery by the stent graft. Fluoroscopic images from elective EVAR procedures performed at a single institution and from open-access sources were selected. Annotations of the "No-Go" zone were performed by trained annotators. A 10-fold cross-validation technique was used to evaluate the performance of the model against human annotations. Primary outcomes were intersection-over-union (IoU) and F1 score and secondary outcomes were pixel-wise accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). RESULTS: The AI model was trained using 369 images procured from 110 different patients/videos, including 18 patients/videos (44 images) from open-access sources. For the primary outcomes, IoU and F1 were 0.43 (standard deviation ± 0.29) and 0.53 (±0.32), respectively. For the secondary outcomes, accuracy, sensitivity, specificity, NPV, and PPV were 0.97 (±0.002), 0.51 (±0.34), 0.99 (±0.001). 0.99 (±0.002), and 0.62 (±0.34), respectively. CONCLUSIONS: AI can effectively identify suboptimal areas of stent deployment during EVAR. Further directions include validating the model on datasets from other institutions and assessing its ability to predict optimal stent graft placement and clinical outcomes.

Intraoperative use of balanced crystalloids versus 0.9% saline: a systematic review and meta-analysis of randomised controlled studies.

BACKGROUND: The evidence regarding optimal crystalloid use in the perioperative period remains unclear. As the primary aim of this study, we sought to summarise the data from RCTs examining whether use of balanced crystalloids compared with 0.9% saline (saline) leads to differences in patient-important outcomes. METHODS: We searched Ovid MEDLINE, Embase, the Cochrane library, and Clinicaltrials.gov, from inception until December 15, 2022, and included RCTs that intraoperatively randomised adult participants to receive either balanced fluids or saline. We pooled data using a random-effects model and present risk ratios (RRs) or mean differences (MDs), along with 95% confidence intervals (CIs). We assessed individual study risk of bias using the modified Cochrane tool, and certainty of evidence using GRADE. RESULTS: Of 5959 citations, we included 38 RCTs (n=3776 patients). Pooled analysis showed that intraoperative use of balanced fluids compared with saline had an uncertain effect on postoperative mortality analysed at the longest point of follow-up (RR 1.51, 95% CI: 0.42-5.36) and postoperative need for renal replacement therapy (RR 0.95, 95% CI: 0.56-1.59), both very low certainty. Furthermore, use of balanced crystalloids probably leads to a higher postoperative serum pH (MD 0.05, 95% CI: 0.04-0.06), moderate certainty. CONCLUSIONS: Use of balanced crystalloids, compared with saline, in the perioperative setting has an uncertain effect on mortality and need for renal replacement therapy but probably improves postoperative acid-base status. Further research is needed to determine whether balanced crystalloid use affects patient-important outcomes. CLINICAL TRIAL REGISTRATION: CRD42022367593.

Evaluating YouTube as a Source of Education for Patients Undergoing Surgery: A Systematic Review.

OBJECTIVE: The objective of this systematic review is to characterize the peer-reviewed literature investigating YouTube as a source of patient education for patients undergoing surgery. SUMMARY BACKGROUND DATA: YouTube is the largest online video sharing platform and has become a substantial source of health information that patients are likely to access before surgery, yet there has been no systematic assessment of peer-reviewed studies. A comprehensive literature search was conducted using EMBASE, MEDLINE, and Ovid HealthStar from inception through to December of 2021. METHODS: All primary studies evaluating YouTube as a source of patient education relating to surgical procedures (general, cardiac, urology, otolaryngology, plastic, vascular) were included. Study screening and data extraction occurred in duplicate with two reviewers. Characteristics extracted included video length, view count, upload source, overall video educational quality, and quality of individual studies. RESULTS: Among 6,453 citations, 56 studies were identified that examined 6,797 videos with 547 hours of content and 1.39 billion views. There were 49 studies that evaluated the educational quality of the videos. A total of 43 quality assessment tools were used, with each study using a mean of 1.90 assessment tools. Per the global rating for assessments, 34/49 studies (69%) concluded that the overall quality of educational content was poor. CONCLUSIONS: While the impact of non-peer-reviewed YouTube videos on patient knowledge for surgery is unclear, the large amount of online content suggests that they are in demand. The overall educational content of these videos is poor, however, and there is substantial heterogeneity in the quality assessment tools used in their evaluation. A peer-reviewed and standardized approach to online education with video content is needed to better support patients.

Fragility of statistically significant outcomes in randomized trials comparing bariatric surgeries.

BACKGROUND: Randomized controlled trials (RCTs) are regarded as high-level evidence, but the strength of their P values can be difficult to ascertain. The Fragility Index (FI) is a novel metric that evaluates the frailty of trial findings. It is defined as the minimum number of patients required to change from a non-event to event for the findings to lose statistical significance. This study aims to characterize the robustness of bariatric surgery RCTs by examining their FIs. METHODS: A search was conducted in MEDLINE, EMBASE, and CENTRAL from January 2000 to February 2022 for RCTs comparing two bariatric surgeries with statistically significant dichotomous outcomes. Bivariate correlation was conducted to identify associations between FI and trial characteristics. RESULTS: A total of 35 RCTs were included with a median sample size of 80 patients (Interquartile range [IQR] 58-109). The median FI was 2 (IQR 0-5), indicating that altering the status of two patients in one treatment arm would overturn the statistical significance of results. Subgroup analyses of RCTs evaluating diabetes-related outcomes produced a FI of 4 (IQR 2-6.5), while RCTs comparing Roux-en-Y gastric bypass and sleeve gastrectomy had an FI of 2 (IQR 0.5-5). Increasing FI was found to be correlated with decreasing P value, increasing sample size, increasing number of events, and increasing journal impact factor. CONCLUSION: Bariatric surgery RCTs are fragile, with only a few patients required to change from non-events to events to reverse the statistical significance of most trials. Future research should examine the use of FI in trial design.

Fragility of statistically significant findings from randomized trials in comparing laparoscopic versus robotic abdominopelvic surgeries.

BACKGROUND: Utility of robotic over laparoscopic approach has been an area of debate across all surgical specialties over the past decade. The fragility index (FI) is a metric that evaluates the frailty of randomized controlled trials (RCTs) findings by altering the status of patients from an event to non-event until significance is lost. This study aims to evaluate the robustness of RCTs comparing laparoscopic and robotic abdominopelvic surgeries through the FI. METHODS: A search was conducted in MEDLINE and EMBASE for RCTs with dichotomous outcomes comparing laparoscopic and robot-assisted surgery in general surgery, gynecology, and urology. The FI and reverse fragility Index (RFI) metrics were used to assess the strength of findings reported by RCTs, and bivariate correlation was conducted to analyze relationships between FI and trial characteristics. RESULTS: A total of 21 RCTs were included, with a median sample size of 89 participants (Interquartile range [IQR] 62-126). The median FI was 2 (IQR 0-15) and median RFI 5.5 (IQR 4-8.5). The median FI was 3 (IQR 1-15) for general surgery (n = 7), 2 (0.5-3.5) for gynecology (n = 4), and 0 (IQR 0-8.5) for urology RCTs (n = 4). Correlation was found between increasing FI and decreasing p-value, but not sample size, number of outcome events, journal impact factor, loss to follow-up, or risk of bias. CONCLUSION: RCTs comparing laparoscopic and robotic abdominal surgery did not prove to be very robust. While possible advantages of robotic surgery may be emphasized, it remains novel and requires further concrete RCT data.

Completeness of reporting in systematic reviews and meta-analyses in vascular surgery.

OBJECTIVE: Systematic reviews (SRs) and meta-analyses are essential in informing clinical guidelines and decision-making. Complete reporting of SRs through compliance to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines promotes transparency, reproducibility, and consistency across the literature. The purpose of this meta-epidemiological study is to assess the completeness of reporting of SRs in the vascular surgery literature. METHODS: MEDLINE and Embase were used to search through four major vascular surgery journals and four high impact general medical journals for SRs published between 2018 and October 2022 evaluating clinical treatments for any pathology treated by a vascular surgeon. Data screening and extraction were conducted in duplicate. The reporting completeness of each included SR was measured with reference to the 27-item PRISMA checklist, and methodological quality was evaluated using the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR 2) tool. Parametric tests were used to evaluate for associations between PRISMA score and study funding, protocol registration, publication in a higher impact factor journal, and AMSTAR 2 score. The protocol is available online: 10.17605/OSF.IO/VBC5N. RESULTS: Of 1653 articles captured in the initial search, 162 SRs were included in the final analysis. All SRs had more than one incomplete PRISMA item. The mean PRISMA score was 21.2/27 (standard deviation: 2.9, 78.5% compliance), and the mean AMSTAR 2 score was 11.7/16 (standard deviation: 1.9, 73%). SRs that had a prospectively registered protocol had a higher PRISMA score (22.9 vs 20.6, P < .001) as did those that were published in higher impact factor journals (23.3 vs 21.0, P = .017). There was a large positive correlation between an SR's PRISMA and AMSTAR 2 scores (Pearson r = 0.655, 95% confidence interval: 0.55-0.74). There were no associations between the PRISMA score and publication year (P = .067) or funding status (P = .076). CONCLUSIONS: Overall, the reporting of SRs and meta-analyses in vascular surgery is less than ideal, with several key items being consistently under-reported. Prospective registration and methodological quality as measured by AMSTAR 2 scores are positively associated with improved reporting. Authors, reviewers, and journal editors should consider these findings moving forward to encourage completeness of SR reporting. Raising awareness surrounding the value of complete reporting of SRs can aid in enhancing the quality of evidence, and journals should consider these findings in methods used to promote SR reporting.

Assessing the robustness of negative vascular surgery randomized controlled trials using their reverse fragility index.

OBJECTIVE: The reverse fragility index (RFI) describes the number of event conversions needed to convert a statistically nonsignificant dichotomous outcome to a significant one. The objective of the present study was to assess the RFI of vascular surgery randomized controlled trials (RCTs) comparing endovascular vs open surgery for the treatment of abdominal aortic aneurysms (AAAs), carotid artery stenosis (CAS), and peripheral artery disease (PAD). METHODS: MEDLINE and Embase were searched for RCTs that had investigated AAAs, CAS, or PAD with statistically nonsignificant binary primary outcomes. The primary outcome for the present study was the median RFI. Calculation of the RFI was performed by creating two-by-two contingency tables and subtracting events from the group with fewer events and adding nonevents to the same group until a two-tailed Fisher exact test had produced a statistically significant result (P ≤ .05). RESULTS: Of 4187 reports, 49 studies reporting 103 different primary end points were included. The overall median RFI was 7 (interquartile range [IQR], 5-13). The specific RFIs for AAA, CAS, and PAD were 10 (IQR, 6-15.5), 6 (IQR, 5-9.5), and 7 (IQR, 5.5-10), respectively. Of the 103 end points, 42 (47%) had had a loss to follow-up greater than the RFI, of which 10 were AAA trials (24%), 23 were CAS trials (55%), and 9 were PAD trials (21%). The Pearson correlation demonstrated a significant positive relationship between a study's RFI and the impact factor of its publishing journal (r = 0.38; 95% confidence interval [CI], 0.20-0.54; P < .01), length of follow-up (r = 0.43; 95% CI, 0.26-0.58; P < .01), and sample size (r = 0.28; 95% CI, 0.09-0.45; P < .01). CONCLUSIONS: A small number of events (median, 7) was required to change the outcome of negative RCTs from statistically nonsignificant to significant, with 47% of the studies having missing data that could have reversed the finding of its primary outcome. Reporting of the RFI relative to the loss to follow-up could be of benefit in future trials and provide confidence regarding the robustness of the P value.

A Systematic Review and Bibliometric Analysis of Applications of Artificial Intelligence and Machine Learning in Vascular Surgery.

BACKGROUND: Artificial intelligence (AI) and machine learning (ML) have seen increasingly intimate integration with medicine and healthcare in the last 2 decades. The objective of this study was to summarize all current applications of AI and ML in the vascular surgery literature and to conduct a bibliometric analysis of published studies. METHODS: A comprehensive literature search was conducted through Embase, MEDLINE, and Ovid HealthStar from inception until February 19, 2021. Reporting of this study was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Title and abstract screening, full-text screening, and data extraction were conducted in duplicate. Data extracted included study metadata, the clinical area of study within vascular surgery, type of AI/ML method used, dataset, and the application of AI/ML. Publishing journals were classified as having either a clinical scope or technical scope. The author academic background was classified as clinical, nonclinical (e.g., engineering), or both, depending on author affiliation. RESULTS: The initial search identified 7,434 studies, of which 249 were included for a final analysis. The rate of publications is exponentially increasing, with 158 (63%) studies being published in the last 5 years alone. Studies were most commonly related to carotid artery disease (118, 47%), abdominal aortic aneurysms (51, 20%), and peripheral arterial disease (26, 10%). Study authors employed an average of 1.50 (range: 1-6) distinct AI methods in their studies. The application of AI/ML methods broadly related to predictive models (54, 22%), image segmentation (49, 19.4%), diagnostic methods (46, 18%), or multiple combined applications (91, 37%). The most commonly used AI/ML methods were artificial neural networks (155/378 use cases, 41%), support vector machines (64, 17%), k-nearest neighbors algorithm (26, 7%), and random forests (23, 6%). Datasets to which these AI/ML methods were applied frequently involved ultrasound images (87, 35%), computed tomography (CT) images (42, 17%), clinical data (34, 14%), or multiple datasets (36, 14%). Overall, 22 (9%) studies were published in journals specific to vascular surgery, with the majority (147/249, 59%) being published in journals with a scope related to computer science or engineering. Among 1,576 publishing authors, 46% had exclusively a clinical background, 48% a nonclinical background, and 5% had both a clinical and nonclinical background. CONCLUSIONS: There is an exponentially growing body of literature describing the use of AI and ML in vascular surgery. There is a focus on carotid artery disease and abdominal aortic disease, with many other areas of vascular surgery under-represented. Neural networks and support vector machines composed most AI methods in the literature. As AI/ML continue to see expanded applications in the field, it is important that vascular surgeons appreciate its potential and limitations. In addition, as it sees increasing use, there is a need for clinicians with expertise in AI/ML methods who can optimize its transition into daily practice.

Quality of Reporting on Guideline, Protocol, or Algorithm Implementation in Adult Trauma Centers: A Systematic Review.

OBJECTIVE: To appraise the quality of reporting on guideline, protocol, and algorithm implementations in adult trauma settings according to the Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0). BACKGROUND: At present we do not know if published reports of guideline implementations in trauma settings are of sufficient quality to facilitate replication by other centers wishing to implement the same or similar guidelines. METHODS: A systematic review of the literature was conducted. Articles were identified through electronic databases and hand searching relevant trauma journals. Studies meeting inclusion criteria focused on a guideline, protocol, or algorithm that targeted adult trauma patients ≥18 years and/or trauma patient care providers, and evaluated the effectiveness of guideline, protocol, or algorithm implementation in terms of change in clinical practice or patient outcomes. Each included study was assessed in duplicate for adherence to the 18-item SQUIRE 2.0 criteria. The primary endpoint was the proportion of studies meeting at least 80% (score ≥15) of SQUIRE 2.0. RESULTS: Of 7368 screened studies, 74 met inclusion criteria. Thirty-nine percent of studies scored ≥80% on SQUIRE 2.0. Criteria that were met most frequently were abstract (93%), problem description (93%), and specific aims (89%). The lowest scores appeared in the funding (28%), context (47%), and results (54%) criteria. No study indicated using SQUIRE 2.0 as a guideline to writing the report. CONCLUSIONS: Significant opportunity exists to improve the utility of guideline implementation reports in adult trauma settings, particularly in the domains of study context and the implications of context for study outcomes.

A Systematic Review of Economic Evaluations in Vascular Surgery.

BACKGROUND: With increasing healthcare costs and the emergence of new technologies in vascular surgery, economic evaluations play a critical role in informing decision-making that optimizes patient outcomes while minimizing per capita costs. The objective of this systematic review is to describe all English published economic evaluations in vascular surgery and to identify any significant gaps in the literature. METHODS: We conducted a comprehensive English literature review of EMBASE, MEDLINE, The Cochrane Library, Ovid Health Star, and Business Source Complete from inception until December 1, 2018. Two independent reviewers screened articles for eligibility using predetermined inclusion criteria and subsequently extracted data. Articles were included if they compared 2 or more vascular surgery interventions using either a partial economic evaluation (cost analysis) or full economic evaluation (cost-utility, cost-benefit, and/or cost-effectiveness analysis). Data extracted included publishing journal, date of publication, country of origin of authors, type of economic evaluation, and domain of vascular surgery. RESULTS: A total of 234 papers were included in the analysis. The majority of the papers included only a cost analysis (183, 78%), and there were only 51 papers that conducted a full economic analysis (22%). The 51 papers conducted a total of 69 economic analyses. This consisted of 32 cost-effectiveness analyses, 29 cost-utility analyses, and 8 cost-benefit analyses. The most common domains studied were aneurysmal disease (89, 38%) and peripheral vascular disease (50, 21%). Economic evaluations were commonly published in the Journal of Vascular Surgery (83, 35%) and Annals of Vascular Surgery (32, 14%), with most study authors located in the United States (127, 54%). There was a trend of economic evaluations being published more frequently in recent years. CONCLUSIONS: The majority of vascular surgery economic evaluations used only a cost analysis, rather than a full economic evaluation, which may not be ideal in pursuing interventions that simultaneously optimize cost and patient outcomes. The literature is lacking in full economic evaluations-a trend persistent in other surgical specialties-and there is a need for full economic evaluations to be conducted in the field of vascular surgery.

Factors relating to the publication rate of abstracts presented from 2012 to 2015 at the Society for Vascular Surgery Annual Meeting.

OBJECTIVE: The Society for Vascular Surgery's Annual Meeting acts as a means of disseminating research findings among vascular surgeons through the presentation of research abstracts. Following presentation at the meeting, research is often compiled into a full-text manuscript and submitted to a peer-reviewed journal. However, not all abstracts accepted to the Vascular Annual Meeting (VAM) eventually have a corresponding full-text publication. The objectives of this study were to establish the publication rate of abstracts presented between 2012 and 2015 to the VAM and to identify factors correlating with publication status. METHODS: Abstracts presented at the VAM were available through the Journal of Vascular Surgery. Data extracted from eligible abstracts included level of evidence according to the Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence scheme, type of study (prognostic, therapeutic/harm, diagnostic), sample size, and status of outcome (positive, negative, or descriptive findings). Publication status of the abstracts was determined through a comprehensive literature review of PubMed (MEDLINE), Ovid (MEDLINE), and Embase. A multivariable logistic regression was conducted to determine factors correlating with publication status. RESULTS: The publication rate during the study period was 43.0% with a median time to publication of 9 months, with 412 of the 958 abstracts having a corresponding full-text publication in 48 journals with weighted mean impact factor of 3.40. Eleven journals collectively published 372 (90.3%) of the articles, with the Journal of Vascular Surgery publishing 280 (68.0%) of the manuscripts. Our logistic regression model demonstrated that factors positively affecting publication status were a positive status of outcome (odds ratio, 2.59; 95% confidence interval, 1.56-4.28) and a logarithmic increase in the sample size of the study (odds ratio, 1.35; 95% confidence interval, 1.13-1.60). In addition, studies with a corresponding full-text publication had a greater median sample size (250) compared with those without one (143; P < .001). CONCLUSIONS: From 2012 to 2015, 43.0% of VAM abstracts had a corresponding full-text publication, with greater sample size and a positive status of outcome positively correlating with likelihood of publication. Studies with negative findings made up a small proportion of conference abstracts (9.6%) and were the least likely to be published. Given the relatively small size of the specialty of vascular surgery, it may be particularly important to be mindful of publication bias. It may be worthwhile to give additional consideration to acceptance of abstracts or publication of studies with negative results that meaningfully contribute to the literature.

Level of clinical evidence presented at the Society for Vascular Surgery Annual Meeting during a 5-year period (2012-2016).

OBJECTIVE: During the past decades, there has been an increasing emphasis on the use of high-quality evidence to inform clinical decision-making. The purpose of our study was to assess trends in the level of evidence (LOE) of abstracts presented at the Vascular Annual Meeting from 2012 to 2016. METHODS: All Vascular Annual Meeting abstracts for 2012 to 2016 were obtained through the Journal of Vascular Surgery. Two reviewers independently screened abstracts for eligibility. Research with a nonclinical focus was excluded from the study. Data extracted from eligible abstracts included study type (therapeutic, prognostic, diagnostic), study size, country of academic institution of primary author, presentation type, and whether the sample was recruited or from a database. Abstracts were assigned an LOE using the 2011 Oxford Centre for Evidence-Based Medicine classification scheme based on study design (eg, case series, randomized controlled trial). A χ2 test and analysis of variance test were conducted to assess nonrandom changes in LOE during the study period. RESULTS: Of the 1403 abstracts screened, 1147 were included. Inter-rater agreement was high (κ value for abstract screening was 0.93; κ value for data extraction was 0.89). Therapeutic studies were the most common study type (58%), followed by prognostic studies (37%), then diagnostic studies (5%). The majority of abstracts (75.0%) were submitted from North American institutions. Overall, 0.35% of the presentations were level I evidence, 3.1% level II, 52.8% level III, 38.0% level IV, and 5.7% level V. The average LOE per year fluctuated between 3.54 and 3.32, with a mean LOE of 3.45. The proportion of high-quality evidence (level I and level II) increased in the years 2015 and 2016, representing 78% of all level I and level II abstracts presented in the 5-year period. A χ2 test between LOE and year yielded a P value of .0084, indicating significant nonrandom change in LOE between 2012 and 2016. The majority of high LOE research was presented in poster sessions (37.5%), plenary sessions (27.5%), and international forum sessions/talks (25%) at the meeting. CONCLUSIONS: Overall, average LOE remained relatively consistent between 2012 and 2016, with most abstracts classified as level III or level IV. There was a gradual, albeit minor, increase in the proportion of level I and level II evidence in 2015 and 2016, potentially indicating the increasing commitment to producing and disseminating high-level research in vascular surgery. Furthermore, a lack of a classification tool specific to vascular surgery research occasionally presented a challenge in assigning LOE, perhaps indicating a need for such a tool in this specialty.