Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index.

OBJECTIVE: The fragility index (FI) measures the robustness of statistically significant findings in randomized controlled trials (RCTs) by quantifying the minimum number of event conversions required to reverse a dichotomous outcome's statistical significance. In vascular surgery, many clinical guidelines and critical decision-making points are informed by a handful of key RCTs, especially regarding open surgical versus endovascular treatment. The objective of this study is to evaluate the FI of RCTs with statistically significant primary outcomes that compared open vs endovascular surgery in vascular surgery. METHODS: In this meta-epidemiological study and systematic review, MEDLINE, Embase, and CENTRAL were searched for RCTs comparing open versus endovascular treatments for abdominal aortic aneurysms, carotid artery stenosis, and peripheral arterial disease to December 2022. RCTs with statistically significant primary outcomes were included. Data screening and extraction were conducted in duplicate. The FI was calculated by adding an event to the group with the smaller number of events while subtracting a nonevent to the same group until Fisher's exact test produced a nonstatistically significant result. The primary outcome was the FI and proportion of outcomes where the loss to follow-up was greater than the FI. The secondary outcomes assessed the relationship of the FI to disease state, presence of commercial funding, and study design. RESULTS: Overall, 5133 articles were captured in the initial search with 21 RCTs reporting 23 different primary outcomes being included in the final analysis. The median FI (first quartile, third quartile) was 3 (3, 20) with 16 outcomes (70%) reporting a loss to follow-up greater than its FI. Mann-Whitney U test revealed that commercially funded RCTs and composite outcomes had greater FIs (median, 20.0 [5.5, 24.5] vs median, 3.0 [2.0, 5.5], P = .035; median, 21 [8, 38] vs median, 3.0 [2.0, 8.5], P = .01, respectively). The FI did not vary between disease states (P = .285) or between index and follow-up trials (P = .147). There were significant correlations between the FI and P values (Pearson r = 0.90; 95% confidence interval, 0.77-0.96), and the number of events (r = 0.82; 95% confidence interval, 0.48-0.97). CONCLUSIONS: A small number of event conversions (median, 3) are needed to alter the statistical significance of primary outcomes in vascular surgery RCTs evaluating open surgical and endovascular treatments. Most studies had loss to follow-up greater than its FI, which can call into question trial results, and commercially funded studies had a greater FI. The FI and these findings should be considered in future trial design in vascular surgery.

Evaluating the Appropriateness, Consistency, and Readability of ChatGPT in Critical Care Recommendations.

Background: We assessed 2 versions of the large language model (LLM) ChatGPT-versions 3.5 and 4.0-in generating appropriate, consistent, and readable recommendations on core critical care topics. Research Question: How do successive large language models compare in terms of generating appropriate, consistent, and readable recommendations on core critical care topics? Design and Methods: A set of 50 LLM-generated responses to clinical questions were evaluated by 2 independent intensivists based on a 5-point Likert scale for appropriateness, consistency, and readability. Results: ChatGPT 4.0 showed significantly higher median appropriateness scores compared to ChatGPT 3.5 (4.0 vs 3.0, P < .001). However, there was no significant difference in consistency between the 2 versions (40% vs 28%, P = 0.291). Readability, assessed by the Flesch-Kincaid Grade Level, was also not significantly different between the 2 models (14.3 vs 14.4, P = 0.93). Interpretation: Both models produced "hallucinations"-misinformation delivered with high confidence-which highlights the risk of relying on these tools without domain expertise. Despite potential for clinical application, both models lacked consistency producing different results when asked the same question multiple times. The study underscores the need for clinicians to understand the strengths and limitations of LLMs for safe and effective implementation in critical care settings. Registration: https://osf.io/8chj7/.

Development of an Artificial Intelligence Tool for Intraoperative Guidance During Endovascular Abdominal Aortic Aneurysm Repair.

BACKGROUND: Adverse events during surgery can occur in part due to errors in visual perception and judgment. Deep learning is a branch of artificial intelligence (AI) that has shown promise in providing real-time intraoperative guidance. This study aims to train and test the performance of a deep learning model that can identify inappropriate landing zones during endovascular aneurysm repair (EVAR). METHODS: A deep learning model was trained to identify a "No-Go" landing zone during EVAR, defined by coverage of the lowest renal artery by the stent graft. Fluoroscopic images from elective EVAR procedures performed at a single institution and from open-access sources were selected. Annotations of the "No-Go" zone were performed by trained annotators. A 10-fold cross-validation technique was used to evaluate the performance of the model against human annotations. Primary outcomes were intersection-over-union (IoU) and F1 score and secondary outcomes were pixel-wise accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). RESULTS: The AI model was trained using 369 images procured from 110 different patients/videos, including 18 patients/videos (44 images) from open-access sources. For the primary outcomes, IoU and F1 were 0.43 (standard deviation ± 0.29) and 0.53 (±0.32), respectively. For the secondary outcomes, accuracy, sensitivity, specificity, NPV, and PPV were 0.97 (±0.002), 0.51 (±0.34), 0.99 (±0.001). 0.99 (±0.002), and 0.62 (±0.34), respectively. CONCLUSIONS: AI can effectively identify suboptimal areas of stent deployment during EVAR. Further directions include validating the model on datasets from other institutions and assessing its ability to predict optimal stent graft placement and clinical outcomes.

YouTube as a source of education in perioperative anesthesia for patients and trainees: a systematic review.

BACKGROUND: Online video sharing platforms like YouTube (Google LLC, San Bruno, CA, USA) have become a substantial source of health information. We sought to conduct a systematic review of studies assessing the overall quality of perioperative anesthesia videos on YouTube. METHODS: We searched Embase, MEDLINE, and Ovid for articles published from database inception to 1 May 2023. We included primary studies evaluating YouTube videos as a source of information regarding perioperative anesthesia. We excluded studies not published in English and studies assessing acute or chronic pain. Studies were screened and data were extracted in duplicate by two reviewers. We appraised the quality of studies according to the social media framework published in the literature. We used descriptive statistics to report the results using mean, standard deviation, range, and n/total N (%). RESULTS: Among 8,908 citations, we identified 14 studies that examined 796 videos with 59.7 hr of content and 47.5 million views. Among the 14 studies that evaluated the video content quality, 17 different quality assessment tools were used, only three of which were externally validated (Global Quality Score, modified DISCERN score, and JAMA score). Per global assessment rating of video quality, 11/13 (85%) studies concluded the overall video quality as poor. CONCLUSIONS: Overall, the educational content quality of YouTube videos evaluated in the literature accessible as an educational resource regarding perioperative anesthesia was poor. While these videos are in demand, their impact on patient and trainee education remains unclear. A standardized methodology for evaluating online videos is merited to improve future reporting. A peer-reviewed approach to online open-access videos is needed to support patient and trainee education in anesthesia. STUDY REGISTRATION: Open Science Framework ( https://osf.io/ajse9 ); first posted, 1 May 2023.

RéSUMé: CONTEXTE: Les plateformes de partage de vidéos en ligne comme YouTube (Google LLC, San Bruno, CA, États-Unis) sont devenues une source importante d'informations sur la santé. Nous avons cherché à réaliser une revue systématique des études évaluant la qualité globale des vidéos d'anesthésie périopératoire sur YouTube. MéTHODE: Nous avons recherché des articles dans Embase, MEDLINE et Ovid publiés depuis la création de ces bases de données jusqu'au 1er mai 2023. Nous avons inclus des études primaires évaluant les vidéos YouTube comme source d'information sur l'anesthésie périopératoire. Nous avons exclu les études publiées dans une langue autre que l'anglais et les études évaluant la douleur aiguë ou chronique. Les études ont été examinées et les données ont été extraites en double par deux personnes. Nous avons évalué la qualité des études selon le cadre des médias sociaux publié dans la littérature. Nous avons utilisé des statistiques descriptives pour rapporter les résultats en utilisant la moyenne, l'écart type, la plage et n/total N (%). RéSULTATS: Parmi 8908 citations, nous avons identifié 14 études qui ont examiné 796 vidéos avec 59,7 heures de contenu et 47,5 millions de vues. Parmi les 14 études qui ont évalué la qualité du contenu vidéo, 17 outils d'évaluation de la qualité différents ont été utilisés, dont seulement trois ont été validés en externe (Score Global Quality, score DISCERN modifié et score JAMA). Selon l'évaluation globale de la qualité des vidéos, 11 études sur 13 (85 %) ont conclu que la qualité globale des vidéos était médiocre. CONCLUSION: Dans l'ensemble, la qualité du contenu éducatif des vidéos YouTube évaluées dans la littérature accessible en tant que ressource éducative concernant l'anesthésie périopératoire était médiocre. Bien que ces vidéos soient très demandées, leur impact sur la formation de la patientèle et des stagiaires reste incertain. Une méthodologie normalisée d'évaluation des vidéos en ligne est nécessaire pour améliorer les évaluations futures. Une approche évaluée par les pairs pour les vidéos en libre accès en ligne est nécessaire pour soutenir la formation de la patientèle et des stagiaires en anesthésie. ENREGISTREMENT DE L'éTUDE: Open Science Framework ( https://osf.io/ajse9 ); première publication le 1er mai 2023.

Evaluating YouTube as a Source of Education for Patients Undergoing Surgery: A Systematic Review.

OBJECTIVE: The objective of this systematic review is to characterize the peer-reviewed literature investigating YouTube as a source of patient education for patients undergoing surgery. SUMMARY BACKGROUND DATA: YouTube is the largest online video sharing platform and has become a substantial source of health information that patients are likely to access before surgery, yet there has been no systematic assessment of peer-reviewed studies. A comprehensive literature search was conducted using EMBASE, MEDLINE, and Ovid HealthStar from inception through to December of 2021. METHODS: All primary studies evaluating YouTube as a source of patient education relating to surgical procedures (general, cardiac, urology, otolaryngology, plastic, vascular) were included. Study screening and data extraction occurred in duplicate with two reviewers. Characteristics extracted included video length, view count, upload source, overall video educational quality, and quality of individual studies. RESULTS: Among 6,453 citations, 56 studies were identified that examined 6,797 videos with 547 hours of content and 1.39 billion views. There were 49 studies that evaluated the educational quality of the videos. A total of 43 quality assessment tools were used, with each study using a mean of 1.90 assessment tools. Per the global rating for assessments, 34/49 studies (69%) concluded that the overall quality of educational content was poor. CONCLUSIONS: While the impact of non-peer-reviewed YouTube videos on patient knowledge for surgery is unclear, the large amount of online content suggests that they are in demand. The overall educational content of these videos is poor, however, and there is substantial heterogeneity in the quality assessment tools used in their evaluation. A peer-reviewed and standardized approach to online education with video content is needed to better support patients.

Fragility of statistically significant outcomes in randomized trials comparing bariatric surgeries.

BACKGROUND: Randomized controlled trials (RCTs) are regarded as high-level evidence, but the strength of their P values can be difficult to ascertain. The Fragility Index (FI) is a novel metric that evaluates the frailty of trial findings. It is defined as the minimum number of patients required to change from a non-event to event for the findings to lose statistical significance. This study aims to characterize the robustness of bariatric surgery RCTs by examining their FIs. METHODS: A search was conducted in MEDLINE, EMBASE, and CENTRAL from January 2000 to February 2022 for RCTs comparing two bariatric surgeries with statistically significant dichotomous outcomes. Bivariate correlation was conducted to identify associations between FI and trial characteristics. RESULTS: A total of 35 RCTs were included with a median sample size of 80 patients (Interquartile range [IQR] 58-109). The median FI was 2 (IQR 0-5), indicating that altering the status of two patients in one treatment arm would overturn the statistical significance of results. Subgroup analyses of RCTs evaluating diabetes-related outcomes produced a FI of 4 (IQR 2-6.5), while RCTs comparing Roux-en-Y gastric bypass and sleeve gastrectomy had an FI of 2 (IQR 0.5-5). Increasing FI was found to be correlated with decreasing P value, increasing sample size, increasing number of events, and increasing journal impact factor. CONCLUSION: Bariatric surgery RCTs are fragile, with only a few patients required to change from non-events to events to reverse the statistical significance of most trials. Future research should examine the use of FI in trial design.

Completeness of reporting in systematic reviews and meta-analyses in vascular surgery.

OBJECTIVE: Systematic reviews (SRs) and meta-analyses are essential in informing clinical guidelines and decision-making. Complete reporting of SRs through compliance to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines promotes transparency, reproducibility, and consistency across the literature. The purpose of this meta-epidemiological study is to assess the completeness of reporting of SRs in the vascular surgery literature. METHODS: MEDLINE and Embase were used to search through four major vascular surgery journals and four high impact general medical journals for SRs published between 2018 and October 2022 evaluating clinical treatments for any pathology treated by a vascular surgeon. Data screening and extraction were conducted in duplicate. The reporting completeness of each included SR was measured with reference to the 27-item PRISMA checklist, and methodological quality was evaluated using the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR 2) tool. Parametric tests were used to evaluate for associations between PRISMA score and study funding, protocol registration, publication in a higher impact factor journal, and AMSTAR 2 score. The protocol is available online: 10.17605/OSF.IO/VBC5N. RESULTS: Of 1653 articles captured in the initial search, 162 SRs were included in the final analysis. All SRs had more than one incomplete PRISMA item. The mean PRISMA score was 21.2/27 (standard deviation: 2.9, 78.5% compliance), and the mean AMSTAR 2 score was 11.7/16 (standard deviation: 1.9, 73%). SRs that had a prospectively registered protocol had a higher PRISMA score (22.9 vs 20.6, P < .001) as did those that were published in higher impact factor journals (23.3 vs 21.0, P = .017). There was a large positive correlation between an SR's PRISMA and AMSTAR 2 scores (Pearson r = 0.655, 95% confidence interval: 0.55-0.74). There were no associations between the PRISMA score and publication year (P = .067) or funding status (P = .076). CONCLUSIONS: Overall, the reporting of SRs and meta-analyses in vascular surgery is less than ideal, with several key items being consistently under-reported. Prospective registration and methodological quality as measured by AMSTAR 2 scores are positively associated with improved reporting. Authors, reviewers, and journal editors should consider these findings moving forward to encourage completeness of SR reporting. Raising awareness surrounding the value of complete reporting of SRs can aid in enhancing the quality of evidence, and journals should consider these findings in methods used to promote SR reporting.

Assessing the robustness of negative vascular surgery randomized controlled trials using their reverse fragility index.

OBJECTIVE: The reverse fragility index (RFI) describes the number of event conversions needed to convert a statistically nonsignificant dichotomous outcome to a significant one. The objective of the present study was to assess the RFI of vascular surgery randomized controlled trials (RCTs) comparing endovascular vs open surgery for the treatment of abdominal aortic aneurysms (AAAs), carotid artery stenosis (CAS), and peripheral artery disease (PAD). METHODS: MEDLINE and Embase were searched for RCTs that had investigated AAAs, CAS, or PAD with statistically nonsignificant binary primary outcomes. The primary outcome for the present study was the median RFI. Calculation of the RFI was performed by creating two-by-two contingency tables and subtracting events from the group with fewer events and adding nonevents to the same group until a two-tailed Fisher exact test had produced a statistically significant result (P ≤ .05). RESULTS: Of 4187 reports, 49 studies reporting 103 different primary end points were included. The overall median RFI was 7 (interquartile range [IQR], 5-13). The specific RFIs for AAA, CAS, and PAD were 10 (IQR, 6-15.5), 6 (IQR, 5-9.5), and 7 (IQR, 5.5-10), respectively. Of the 103 end points, 42 (47%) had had a loss to follow-up greater than the RFI, of which 10 were AAA trials (24%), 23 were CAS trials (55%), and 9 were PAD trials (21%). The Pearson correlation demonstrated a significant positive relationship between a study's RFI and the impact factor of its publishing journal (r = 0.38; 95% confidence interval [CI], 0.20-0.54; P < .01), length of follow-up (r = 0.43; 95% CI, 0.26-0.58; P < .01), and sample size (r = 0.28; 95% CI, 0.09-0.45; P < .01). CONCLUSIONS: A small number of events (median, 7) was required to change the outcome of negative RCTs from statistically nonsignificant to significant, with 47% of the studies having missing data that could have reversed the finding of its primary outcome. Reporting of the RFI relative to the loss to follow-up could be of benefit in future trials and provide confidence regarding the robustness of the P value.

Intraoperative use of balanced crystalloids versus 0.9% saline: a systematic review and meta-analysis of randomised controlled studies.

BACKGROUND: The evidence regarding optimal crystalloid use in the perioperative period remains unclear. As the primary aim of this study, we sought to summarise the data from RCTs examining whether use of balanced crystalloids compared with 0.9% saline (saline) leads to differences in patient-important outcomes. METHODS: We searched Ovid MEDLINE, Embase, the Cochrane library, and Clinicaltrials.gov, from inception until December 15, 2022, and included RCTs that intraoperatively randomised adult participants to receive either balanced fluids or saline. We pooled data using a random-effects model and present risk ratios (RRs) or mean differences (MDs), along with 95% confidence intervals (CIs). We assessed individual study risk of bias using the modified Cochrane tool, and certainty of evidence using GRADE. RESULTS: Of 5959 citations, we included 38 RCTs (n=3776 patients). Pooled analysis showed that intraoperative use of balanced fluids compared with saline had an uncertain effect on postoperative mortality analysed at the longest point of follow-up (RR 1.51, 95% CI: 0.42-5.36) and postoperative need for renal replacement therapy (RR 0.95, 95% CI: 0.56-1.59), both very low certainty. Furthermore, use of balanced crystalloids probably leads to a higher postoperative serum pH (MD 0.05, 95% CI: 0.04-0.06), moderate certainty. CONCLUSIONS: Use of balanced crystalloids, compared with saline, in the perioperative setting has an uncertain effect on mortality and need for renal replacement therapy but probably improves postoperative acid-base status. Further research is needed to determine whether balanced crystalloid use affects patient-important outcomes. CLINICAL TRIAL REGISTRATION: CRD42022367593.

Fragility of statistically significant findings from randomized trials in comparing laparoscopic versus robotic abdominopelvic surgeries.

BACKGROUND: Utility of robotic over laparoscopic approach has been an area of debate across all surgical specialties over the past decade. The fragility index (FI) is a metric that evaluates the frailty of randomized controlled trials (RCTs) findings by altering the status of patients from an event to non-event until significance is lost. This study aims to evaluate the robustness of RCTs comparing laparoscopic and robotic abdominopelvic surgeries through the FI. METHODS: A search was conducted in MEDLINE and EMBASE for RCTs with dichotomous outcomes comparing laparoscopic and robot-assisted surgery in general surgery, gynecology, and urology. The FI and reverse fragility Index (RFI) metrics were used to assess the strength of findings reported by RCTs, and bivariate correlation was conducted to analyze relationships between FI and trial characteristics. RESULTS: A total of 21 RCTs were included, with a median sample size of 89 participants (Interquartile range [IQR] 62-126). The median FI was 2 (IQR 0-15) and median RFI 5.5 (IQR 4-8.5). The median FI was 3 (IQR 1-15) for general surgery (n = 7), 2 (0.5-3.5) for gynecology (n = 4), and 0 (IQR 0-8.5) for urology RCTs (n = 4). Correlation was found between increasing FI and decreasing p-value, but not sample size, number of outcome events, journal impact factor, loss to follow-up, or risk of bias. CONCLUSION: RCTs comparing laparoscopic and robotic abdominal surgery did not prove to be very robust. While possible advantages of robotic surgery may be emphasized, it remains novel and requires further concrete RCT data.

The fragility of significant results from randomized controlled trials in esophageal surgeries.

While randomized controlled trials (RCTs) are regarded as one of the highest forms of clinical research, the robustness of their P values can be difficult to ascertain. Defined as the minimum number of patients in a study arm that would need to be changed from a non-event to event for the findings to lose significance, the Fragility Index is a method for evaluating results from these trials. This study aims to calculate the Fragility Index for trials evaluating perioperative esophagectomy-related interventions to determine the strength of RCTs in this field. MEDLINE and EMBASE were searched for RCTs related to esophagectomy that reported a significant dichotomous outcome. Two reviewers independently screened articles and performed the data extractions with risk of bias assessment. The Fragility Index was calculated using a two-tailed Fisher's exact test. Bivariate correlation was conducted to evaluate associations between the Fragility Index and study characteristics. 41 RCTs were included, and the median sample size was 80 patients [Interquartile range (IQR) 60-161]. Of the included outcomes, 29 (71%) were primary, and 12 (29%) were secondary. The median Fragility Index was 1 (IQR 1-3), meaning that by changing one patient from a non-event to event, the results would become non-significant. Fragility Index was correlated with P value, number of events, and journal impact factor. The RCTs related to esophagectomy did not prove to be robust, as the significance of their results could be changed by altering the outcome status of a handful of patients in one study arm.

A Systematic Review and Bibliometric Analysis of Applications of Artificial Intelligence and Machine Learning in Vascular Surgery.

BACKGROUND: Artificial intelligence (AI) and machine learning (ML) have seen increasingly intimate integration with medicine and healthcare in the last 2 decades. The objective of this study was to summarize all current applications of AI and ML in the vascular surgery literature and to conduct a bibliometric analysis of published studies. METHODS: A comprehensive literature search was conducted through Embase, MEDLINE, and Ovid HealthStar from inception until February 19, 2021. Reporting of this study was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Title and abstract screening, full-text screening, and data extraction were conducted in duplicate. Data extracted included study metadata, the clinical area of study within vascular surgery, type of AI/ML method used, dataset, and the application of AI/ML. Publishing journals were classified as having either a clinical scope or technical scope. The author academic background was classified as clinical, nonclinical (e.g., engineering), or both, depending on author affiliation. RESULTS: The initial search identified 7,434 studies, of which 249 were included for a final analysis. The rate of publications is exponentially increasing, with 158 (63%) studies being published in the last 5 years alone. Studies were most commonly related to carotid artery disease (118, 47%), abdominal aortic aneurysms (51, 20%), and peripheral arterial disease (26, 10%). Study authors employed an average of 1.50 (range: 1-6) distinct AI methods in their studies. The application of AI/ML methods broadly related to predictive models (54, 22%), image segmentation (49, 19.4%), diagnostic methods (46, 18%), or multiple combined applications (91, 37%). The most commonly used AI/ML methods were artificial neural networks (155/378 use cases, 41%), support vector machines (64, 17%), k-nearest neighbors algorithm (26, 7%), and random forests (23, 6%). Datasets to which these AI/ML methods were applied frequently involved ultrasound images (87, 35%), computed tomography (CT) images (42, 17%), clinical data (34, 14%), or multiple datasets (36, 14%). Overall, 22 (9%) studies were published in journals specific to vascular surgery, with the majority (147/249, 59%) being published in journals with a scope related to computer science or engineering. Among 1,576 publishing authors, 46% had exclusively a clinical background, 48% a nonclinical background, and 5% had both a clinical and nonclinical background. CONCLUSIONS: There is an exponentially growing body of literature describing the use of AI and ML in vascular surgery. There is a focus on carotid artery disease and abdominal aortic disease, with many other areas of vascular surgery under-represented. Neural networks and support vector machines composed most AI methods in the literature. As AI/ML continue to see expanded applications in the field, it is important that vascular surgeons appreciate its potential and limitations. In addition, as it sees increasing use, there is a need for clinicians with expertise in AI/ML methods who can optimize its transition into daily practice.

Quality of Reporting on Guideline, Protocol, or Algorithm Implementation in Adult Trauma Centers: A Systematic Review.

OBJECTIVE: To appraise the quality of reporting on guideline, protocol, and algorithm implementations in adult trauma settings according to the Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0). BACKGROUND: At present we do not know if published reports of guideline implementations in trauma settings are of sufficient quality to facilitate replication by other centers wishing to implement the same or similar guidelines. METHODS: A systematic review of the literature was conducted. Articles were identified through electronic databases and hand searching relevant trauma journals. Studies meeting inclusion criteria focused on a guideline, protocol, or algorithm that targeted adult trauma patients ≥18 years and/or trauma patient care providers, and evaluated the effectiveness of guideline, protocol, or algorithm implementation in terms of change in clinical practice or patient outcomes. Each included study was assessed in duplicate for adherence to the 18-item SQUIRE 2.0 criteria. The primary endpoint was the proportion of studies meeting at least 80% (score ≥15) of SQUIRE 2.0. RESULTS: Of 7368 screened studies, 74 met inclusion criteria. Thirty-nine percent of studies scored ≥80% on SQUIRE 2.0. Criteria that were met most frequently were abstract (93%), problem description (93%), and specific aims (89%). The lowest scores appeared in the funding (28%), context (47%), and results (54%) criteria. No study indicated using SQUIRE 2.0 as a guideline to writing the report. CONCLUSIONS: Significant opportunity exists to improve the utility of guideline implementation reports in adult trauma settings, particularly in the domains of study context and the implications of context for study outcomes.

Examination of a Canada-Wide Collaboration Platform for Order Sets: Retrospective Analysis.

BACKGROUND: Knowledge translation and dissemination are some of the main challenges that affect evidence-based medicine. Web 2.0 platforms promote the sharing and collaborative development of content. Executable knowledge tools, such as order sets, are a knowledge translation tool whose localization is critical to its effectiveness but a challenge for organizations to develop independently. OBJECTIVE: This paper describes a Web 2.0 resource, referred to as the collaborative network (TCN), for order set development designed to share executable knowledge (order sets). This paper also analyzes the scope of its use, describes its use through network analysis, and examines the provision and use of order sets in the platform by organizational size. METHODS: Data were collected from Think Research's TxConnect platform. We measured interorganization sharing across Canadian hospitals using descriptive statistics. A weighted chi-square analysis was used to evaluate institutional size to share volumes based on institution size, with post hoc Cramer V score to measure the strength of association. RESULTS: TCN consisted of 12,495 order sets across 683 diagnoses or processes. Between January 2010 and March 2015, a total of 131 health care organizations representing 360 hospitals in Canada downloaded order sets 105,496 times. Order sets related to acute coronary syndrome, analgesia, and venous thromboembolism were most commonly shared. COVID-19 order sets were among the most actively shared, adjusting for order set lifetime. A weighted chi-square analysis showed nonrandom downloading behavior (P<.001), with medium-sized institutions downloading content from larger institutions acting as the most significant driver of this variance (chi-gram=124.70). CONCLUSIONS: In this paper, we have described and analyzed a Web 2.0 platform for the sharing of order set content with significant network activity. The robust use of TCN to access customized order sets reflects its value as a resource for health care organizations when they develop or update their own order sets.

The International Federation for Emergency Medicine report on emergency department crowding and access block: A brief summary.

OBJECTIVE: To develop comprehensive guidance that captures international impacts, causes and solutions related to emergency department (ED) crowding and access block. METHODS: Emergency physicians representing 15 countries from all International Federation of Emergency Medicine (IFEM) regions composed the Task Force. Monthly meetings were held via video-conferencing software to achieve consensus for report content. The report was submitted and approved by the IFEM Board on June 1, 2020. RESULTS: A total of 14 topic dossiers, each relating to an aspect of ED crowding, were researched and completed collaboratively by members of the Task Force. CONCLUSIONS: The IFEM report is a comprehensive document intended to be used in whole or by section to inform and address aspects of ED crowding and access block. Overall, ED crowding is a multifactorial issue requiring systems-wide solutions applied at local, regional, and national levels. Access block is the predominant contributor of ED crowding in most parts of the world.

A Systematic Review of Economic Evaluations in Vascular Surgery.

BACKGROUND: With increasing healthcare costs and the emergence of new technologies in vascular surgery, economic evaluations play a critical role in informing decision-making that optimizes patient outcomes while minimizing per capita costs. The objective of this systematic review is to describe all English published economic evaluations in vascular surgery and to identify any significant gaps in the literature. METHODS: We conducted a comprehensive English literature review of EMBASE, MEDLINE, The Cochrane Library, Ovid Health Star, and Business Source Complete from inception until December 1, 2018. Two independent reviewers screened articles for eligibility using predetermined inclusion criteria and subsequently extracted data. Articles were included if they compared 2 or more vascular surgery interventions using either a partial economic evaluation (cost analysis) or full economic evaluation (cost-utility, cost-benefit, and/or cost-effectiveness analysis). Data extracted included publishing journal, date of publication, country of origin of authors, type of economic evaluation, and domain of vascular surgery. RESULTS: A total of 234 papers were included in the analysis. The majority of the papers included only a cost analysis (183, 78%), and there were only 51 papers that conducted a full economic analysis (22%). The 51 papers conducted a total of 69 economic analyses. This consisted of 32 cost-effectiveness analyses, 29 cost-utility analyses, and 8 cost-benefit analyses. The most common domains studied were aneurysmal disease (89, 38%) and peripheral vascular disease (50, 21%). Economic evaluations were commonly published in the Journal of Vascular Surgery (83, 35%) and Annals of Vascular Surgery (32, 14%), with most study authors located in the United States (127, 54%). There was a trend of economic evaluations being published more frequently in recent years. CONCLUSIONS: The majority of vascular surgery economic evaluations used only a cost analysis, rather than a full economic evaluation, which may not be ideal in pursuing interventions that simultaneously optimize cost and patient outcomes. The literature is lacking in full economic evaluations-a trend persistent in other surgical specialties-and there is a need for full economic evaluations to be conducted in the field of vascular surgery.

Factors relating to the publication rate of abstracts presented from 2012 to 2015 at the Society for Vascular Surgery Annual Meeting.

OBJECTIVE: The Society for Vascular Surgery's Annual Meeting acts as a means of disseminating research findings among vascular surgeons through the presentation of research abstracts. Following presentation at the meeting, research is often compiled into a full-text manuscript and submitted to a peer-reviewed journal. However, not all abstracts accepted to the Vascular Annual Meeting (VAM) eventually have a corresponding full-text publication. The objectives of this study were to establish the publication rate of abstracts presented between 2012 and 2015 to the VAM and to identify factors correlating with publication status. METHODS: Abstracts presented at the VAM were available through the Journal of Vascular Surgery. Data extracted from eligible abstracts included level of evidence according to the Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence scheme, type of study (prognostic, therapeutic/harm, diagnostic), sample size, and status of outcome (positive, negative, or descriptive findings). Publication status of the abstracts was determined through a comprehensive literature review of PubMed (MEDLINE), Ovid (MEDLINE), and Embase. A multivariable logistic regression was conducted to determine factors correlating with publication status. RESULTS: The publication rate during the study period was 43.0% with a median time to publication of 9 months, with 412 of the 958 abstracts having a corresponding full-text publication in 48 journals with weighted mean impact factor of 3.40. Eleven journals collectively published 372 (90.3%) of the articles, with the Journal of Vascular Surgery publishing 280 (68.0%) of the manuscripts. Our logistic regression model demonstrated that factors positively affecting publication status were a positive status of outcome (odds ratio, 2.59; 95% confidence interval, 1.56-4.28) and a logarithmic increase in the sample size of the study (odds ratio, 1.35; 95% confidence interval, 1.13-1.60). In addition, studies with a corresponding full-text publication had a greater median sample size (250) compared with those without one (143; P < .001). CONCLUSIONS: From 2012 to 2015, 43.0% of VAM abstracts had a corresponding full-text publication, with greater sample size and a positive status of outcome positively correlating with likelihood of publication. Studies with negative findings made up a small proportion of conference abstracts (9.6%) and were the least likely to be published. Given the relatively small size of the specialty of vascular surgery, it may be particularly important to be mindful of publication bias. It may be worthwhile to give additional consideration to acceptance of abstracts or publication of studies with negative results that meaningfully contribute to the literature.

Level of clinical evidence presented at the Society for Vascular Surgery Annual Meeting during a 5-year period (2012-2016).

OBJECTIVE: During the past decades, there has been an increasing emphasis on the use of high-quality evidence to inform clinical decision-making. The purpose of our study was to assess trends in the level of evidence (LOE) of abstracts presented at the Vascular Annual Meeting from 2012 to 2016. METHODS: All Vascular Annual Meeting abstracts for 2012 to 2016 were obtained through the Journal of Vascular Surgery. Two reviewers independently screened abstracts for eligibility. Research with a nonclinical focus was excluded from the study. Data extracted from eligible abstracts included study type (therapeutic, prognostic, diagnostic), study size, country of academic institution of primary author, presentation type, and whether the sample was recruited or from a database. Abstracts were assigned an LOE using the 2011 Oxford Centre for Evidence-Based Medicine classification scheme based on study design (eg, case series, randomized controlled trial). A χ2 test and analysis of variance test were conducted to assess nonrandom changes in LOE during the study period. RESULTS: Of the 1403 abstracts screened, 1147 were included. Inter-rater agreement was high (κ value for abstract screening was 0.93; κ value for data extraction was 0.89). Therapeutic studies were the most common study type (58%), followed by prognostic studies (37%), then diagnostic studies (5%). The majority of abstracts (75.0%) were submitted from North American institutions. Overall, 0.35% of the presentations were level I evidence, 3.1% level II, 52.8% level III, 38.0% level IV, and 5.7% level V. The average LOE per year fluctuated between 3.54 and 3.32, with a mean LOE of 3.45. The proportion of high-quality evidence (level I and level II) increased in the years 2015 and 2016, representing 78% of all level I and level II abstracts presented in the 5-year period. A χ2 test between LOE and year yielded a P value of .0084, indicating significant nonrandom change in LOE between 2012 and 2016. The majority of high LOE research was presented in poster sessions (37.5%), plenary sessions (27.5%), and international forum sessions/talks (25%) at the meeting. CONCLUSIONS: Overall, average LOE remained relatively consistent between 2012 and 2016, with most abstracts classified as level III or level IV. There was a gradual, albeit minor, increase in the proportion of level I and level II evidence in 2015 and 2016, potentially indicating the increasing commitment to producing and disseminating high-level research in vascular surgery. Furthermore, a lack of a classification tool specific to vascular surgery research occasionally presented a challenge in assigning LOE, perhaps indicating a need for such a tool in this specialty.

YouTube as a Source of Patient and Trainee Education in Vascular Surgery: A Systematic Review.

Objective: Due to its video based approach, YouTube has become a widely accessed educational resource for patients and trainees. This systematic review characterised and evaluated the peer reviewed literature investigating YouTube as a source of patient or trainee education in vascular surgery. Data sources: A comprehensive literature search was conducted using EMBASE, MEDLINE, and Ovid HealthStar from inception until 19 January 2023. All primary studies and conference abstracts evaluating YouTube as a source of vascular surgery education were included. Review methods: Video educational quality was analysed across several factors, including pathology, video audience, and length. Results: Overall, 24 studies were identified examining 3 221 videos with 123.1 hours of content and 37.1 million views. Studies primarily examined YouTube videos on diabetic foot care (7/24, 29%), peripheral arterial disease (3/24, 13%), carotid artery stenosis (3/24, 13%), varicose veins (3/24, 13%), and abdominal aortic aneurysm (2/24, 8%). Video educational quality was analysed using standardised assessment tools, author generated scoring systems, or global author reported assessment of quality. Six studies assessed videos for trainee education, while 18 studies evaluated videos for patient education. Among the 20 studies which reported on the overall quality of educational content, 10/20 studies deemed it poor, and 10/20 studies considered it fair, with 53% of studies noting poor educational quality for videos intended for patients and 40% of studies noting poor educational quality in videos intended for trainees. Poor quality videos had more views than fair quality videos (mean 27 348, 95% CI 15 154-39 543 views vs. 11 372, 95% CI 3 115-19 629 views, p = .030). Conclusion: The overall educational quality of YouTube videos for vascular surgery patient and trainee education is suboptimal. There is significant heterogeneity in the quality assessment tools used in their evaluation. A standardised approach to online education with a consistent quality assessment tool is required to better support online patient and trainee education in vascular surgery.

Harnessing artificial intelligence in bariatric surgery: comparative analysis of ChatGPT-4, Bing, and Bard in generating clinician-level bariatric surgery recommendations.

BACKGROUND: The formulation of clinical recommendations pertaining to bariatric surgery is essential in guiding healthcare professionals. However, the extensive and continuously evolving body of literature in bariatric surgery presents considerable challenge for staying abreast of latest developments and efficient information acquisition. Artificial intelligence (AI) has the potential to streamline access to the salient points of clinical recommendations in bariatric surgery. OBJECTIVES: The study aims to appraise the quality and readability of AI-chat-generated answers to frequently asked clinical inquiries in the field of bariatric and metabolic surgery. SETTING: Remote. METHODS: Question prompts inputted into AI large language models (LLMs) and were created based on pre-existing clinical practice guidelines regarding bariatric and metabolic surgery. The prompts were queried into 3 LLMs: OpenAI ChatGPT-4, Microsoft Bing, and Google Bard. The responses from each LLM were entered into a spreadsheet for randomized and blinded duplicate review. Accredited bariatric surgeons in North America independently assessed appropriateness of each recommendation using a 5-point Likert scale. Scores of 4 and 5 were deemed appropriate, while scores of 1-3 indicated lack of appropriateness. A Flesch Reading Ease (FRE) score was calculated to assess the readability of responses generated by each LLMs. RESULTS: There was a significant difference between the 3 LLMs in their 5-point Likert scores, with mean values of 4.46 (SD .82), 3.89 (.80), and 3.11 (.72) for ChatGPT-4, Bard, and Bing (P < .001). There was a significant difference between the 3 LLMs in the proportion of appropriate answers, with ChatGPT-4 at 85.7%, Bard at 74.3%, and Bing at 25.7% (P < .001). The mean FRE scores for ChatGPT-4, Bard, and Bing, were 21.68 (SD 2.78), 42.89 (4.03), and 14.64 (5.09), respectively, with higher scores representing easier readability. CONCLUSIONS: LLM-based AI chat models can effectively generate appropriate responses to clinical questions related to bariatric surgery, though the performance of different models can vary greatly. Therefore, caution should be taken when interpreting clinical information provided by LLMs, and clinician oversight is necessary to ensure accuracy. Future investigation is warranted to explore how LLMs might enhance healthcare provision and clinical decision-making in bariatric surgery.