Comparison of Medical Research Abstracts Written by Surgical Trainees and Senior Surgeons or Generated by Large Language Models.

Importance: Artificial intelligence (AI) has permeated academia, especially OpenAI Chat Generative Pretrained Transformer (ChatGPT), a large language model. However, little has been reported on its use in medical research. Objective: To assess a chatbot's capability to generate and grade medical research abstracts. Design, Setting, and Participants: In this cross-sectional study, ChatGPT versions 3.5 and 4.0 (referred to as chatbot 1 and chatbot 2) were coached to generate 10 abstracts by providing background literature, prompts, analyzed data for each topic, and 10 previously presented, unassociated abstracts to serve as models. The study was conducted between August 2023 and February 2024 (including data analysis). Exposure: Abstract versions utilizing the same topic and data were written by a surgical trainee or a senior physician or generated by chatbot 1 and chatbot 2 for comparison. The 10 training abstracts were written by 8 surgical residents or fellows, edited by the same senior surgeon, at a high-volume hospital in the Southeastern US with an emphasis on outcomes-based research. Abstract comparison was then based on 10 abstracts written by 5 surgical trainees within the first 6 months of their research year, edited by the same senior author. Main Outcomes and Measures: The primary outcome measurements were the abstract grades using 10- and 20-point scales and ranks (first to fourth). Abstract versions by chatbot 1, chatbot 2, junior residents, and the senior author were compared and judged by blinded surgeon-reviewers as well as both chatbot models. Five academic attending surgeons from Denmark, the UK, and the US, with extensive experience in surgical organizations, research, and abstract evaluation served as reviewers. Results: Surgeon-reviewers were unable to differentiate between abstract versions. Each reviewer ranked an AI-generated version first at least once. Abstracts demonstrated no difference in their median (IQR) 10-point scores (resident, 7.0 [6.0-8.0]; senior author, 7.0 [6.0-8.0]; chatbot 1, 7.0 [6.0-8.0]; chatbot 2, 7.0 [6.0-8.0]; P = .61), 20-point scores (resident, 14.0 [12.0-7.0]; senior author, 15.0 [13.0-17.0]; chatbot 1, 14.0 [12.0-16.0]; chatbot 2, 14.0 [13.0-16.0]; P = .50), or rank (resident, 3.0 [1.0-4.0]; senior author, 2.0 [1.0-4.0]; chatbot 1, 3.0 [2.0-4.0]; chatbot 2, 2.0 [1.0-3.0]; P = .14). The abstract grades given by chatbot 1 were comparable to the surgeon-reviewers' grades. However, chatbot 2 graded more favorably than the surgeon-reviewers and chatbot 1. Median (IQR) chatbot 2-reviewer grades were higher than surgeon-reviewer grades of all 4 abstract versions (resident, 14.0 [12.0-17.0] vs 16.9 [16.0-17.5]; P = .02; senior author, 15.0 [13.0-17.0] vs 17.0 [16.5-18.0]; P = .03; chatbot 1, 14.0 [12.0-16.0] vs 17.8 [17.5-18.5]; P = .002; chatbot 2, 14.0 [13.0-16.0] vs 16.8 [14.5-18.0]; P = .04). When comparing the grades of the 2 chatbots, chatbot 2 gave higher median (IQR) grades for abstracts than chatbot 1 (resident, 14.0 [13.0-15.0] vs 16.9 [16.0-17.5]; P = .003; senior author, 13.5 [13.0-15.5] vs 17.0 [16.5-18.0]; P = .004; chatbot 1, 14.5 [13.0-15.0] vs 17.8 [17.5-18.5]; P = .003; chatbot 2, 14.0 [13.0-15.0] vs 16.8 [14.5-18.0]; P = .01). Conclusions and Relevance: In this cross-sectional study, trained chatbots generated convincing medical abstracts, undifferentiable from resident or senior author drafts. Chatbot 1 graded abstracts similarly to surgeon-reviewers, while chatbot 2 was less stringent. These findings may assist surgeon-scientists in successfully implementing AI in medical research.

Emergency umbilical hernia management: scoping review.

BACKGROUND: Umbilical hernias, while frequently asymptomatic, may become acutely symptomatic, strangulated or obstructed, and require emergency treatment. Robust evidence is required for high-quality care in this field. This scoping review aims to elucidate evidence gaps regarding emergency care of umbilical hernias. METHODS: EMBASE, MEDLINE and CENTRAL databases were searched using a predefined strategy until November 2023. Primary research studies reporting on any aspect of emergency umbilical hernia care and published in the English language were eligible for inclusion. Studies were excluded where emergency umbilical hernia care was not the primary focus and subsets of relevant data were unable to be extracted. Two independent reviewers screened abstracts and full texts, resolving disagreements by consensus or a third reviewer. Data were charted according to core concepts addressed by each study and a narrative synthesis was performed. RESULTS: Searches generated 534 abstracts, from which 32 full texts were assessed and 14 included in the final review. This encompassed 52 042 patients undergoing emergency umbilical hernia care. Most were retrospective cohort designs (11/14), split between single (6/14) and multicentre (8/14) with only one randomized trial. Most multicentre studies were from national databases (7/8). Themes arising included risk assessment (n = 4), timing of surgery (n = 4), investigations (n = 1), repair method (n = 8, four mesh versus suture; four laparoscopic versus open) and operative outcomes (n = 11). The most commonly reported outcomes were mortality (n = 9) and morbidity (n = 7) rates and length of hospital stay (n = 5). No studies included patient-reported outcomes specific to emergency umbilical hernia repair. CONCLUSION: This scoping review demonstrates the paucity of high-quality data for this condition. There is a need for randomized trials addressing all aspects of emergency umbilical hernia repair, with patient-reported outcomes.

EHS Rapid Guideline: Evidence-Informed European Recommendations on Parastomal Hernia Prevention-With ESCP and EAES Participation.

Background: Growing evidence on the use of mesh as a prophylactic measure to prevent parastomal hernia and advances in guideline development methods prompted an update of a previous guideline on parastomal hernia prevention. Objective: To develop evidence-based, trustworthy recommendations, informed by an interdisciplinary panel of stakeholders. Methods: We updated a previous systematic review on the use of a prophylactic mesh for end colostomy, and we synthesized evidence using pairwise meta-analysis. A European panel of surgeons, stoma care nurses, and patients developed an evidence-to-decision framework in line with GRADE and Guidelines International Network standards, moderated by a certified guideline methodologist. The framework considered benefits and harms, the certainty of the evidence, patients' preferences and values, cost and resources considerations, acceptability, equity and feasibility. Results: The certainty of the evidence was moderate for parastomal hernia and low for major morbidity, surgery for parastomal hernia, and quality of life. There was unanimous consensus among panel members for a conditional recommendation for the use of a prophylactic mesh in patients with an end colostomy and fair life expectancy, and a strong recommendation for the use of a prophylactic mesh in patients at high risk to develop a parastomal hernia. Conclusion: This rapid guideline provides evidence-informed, interdisciplinary recommendations on the use of prophylactic mesh in patients with an end colostomy. Further, it identifies research gaps, and discusses implications for stakeholders, including overcoming barriers to implementation and specific considerations regarding validity.

Update Systematic Review, Meta-Analysis and GRADE Assessment of the Evidence on Parastomal Hernia Prevention-A EHS, ESCP and EAES Collaborative Project.

Objective: To perform a systematic review and meta-analysis on the effectiveness of prophylactic mesh for the prevention of parastomal hernia in end colostomy, with the ultimate objective to summarize the evidence for an interdisciplinary, European rapid guideline. Methods: We updated a previous systematic review with de novo evidence search of PubMed from inception up to June 2022. Primary outcome was quality of life (QoL). Secondary outcomes were clinical diagnosis of parastomal hernia, surgery for parastomal hernia, and 30 day or in-hospital complications Clavien-Dindo ≥3. We utilised the revised Cochrane Tool for randomised trials (RoB 2 tool) for risk of bias assessment in the included studies. Minimally important differences were set a priori through voting of the panel members. We appraised the evidence using GRADE and we developed GRADE evidence tables. Results: We included 12 randomized trials. Meta-analysis suggested no difference in QoL between prophylactic mesh and no mesh for primary stoma construction (SMD = 0.03, 95% CI [-0.14 to 0.2], I2 = 0%, low certainty of evidence). With regard to parastomal hernia, the use of prophylactic synthetic mesh resulted in a significant risk reduction of the incidence of the event, according to data from all available randomized trials, irrespective of the follow-up period (OR = 0.33, 95% CI [0.18-0.62], I2 = 74%, moderate certainty of evidence). Sensitivity analyses according to follow-up period were in line with the primary analysis. Little to no difference in surgery for parastomal hernia was encountered after pooled analysis of 10 randomised trials (OR = 0.52, 95% CI [0.25-1.09], I2 = 14%). Finally, no significant difference was found in Clavien-Dindo grade 3 and 4 adverse events after surgery with or without the use of a prophylactic mesh (OR = 0.77, 95% CI [0.45-1.30], I2 = 0%, low certainty of evidence). Conclusion: Prophylactic synthetic mesh placement at the time of permanent end colostomy construction is likely associated with a reduced risk for parastomal hernia and may confer similar risk of peri-operative major morbidity compared to no mesh placement. There may be no difference in quality of life and surgical repair of parastomal hernia with the use of either approach.