Evaluation of a chat GPT generated patient information leaflet about laparoscopic cholecystectomy.

BACKGROUND: Artificial intelligence is increasingly being used in all aspects of life in information compilation and writing, and this includes healthcare. This study aimed to evaluate a Chat GPT generated patient information leaflet (PIL) against a surgeon generated version, in order to explore a potential application of this artificial intelligence language processing model. METHODS: Cross-sectional study, undertaken May to June 2023, asking two cohorts (patients and doctors) to complete a questionnaire evaluating a Chat GPT generated PIL and a surgeon generated PIL about laparoscopic cholecystectomy. The patients were having laparoscopic cholecystectomy at large private Hospital in Melbourne, Australia, and doctors were recruited from this hospital and a public quaternary hospital in Melbourne, Australia. The study included a convenience sample of 28 patients and 16 doctors. The main outcome measure was a questionnaire (maximum score out of 8) based on validated evaluation instrument for PILs. RESULTS: The study recruited 28 patients and 15 doctors to complete the questionnaire. The Chat GPT and surgeon generated PILs were scored similarly by patients (median 8 for both PIL; mean 7.5 for Chat GPT PIL vs. 7.1 for surgeon PIL). Doctors also scored both versions similarly, with slightly higher scores for Chat GPT over surgeon version (median 7 vs. 6; mean 6.7 vs. 5.6, respectively). CONCLUSIONS: The Chat GPT generated PIL was assessed as being as good or slightly better than the surgeon generated version. This study shows that PIL are a feasible application of AI language processing models.

Comparison of risk adjusted excess post-operative pancreatic fistula rates after pancreaticoduodenectomy using cumulative sum analysis.

BACKGROUND: Post-operative pancreatic fistula (POPF) is a key outcome post pancreaticoduodenectomy. There are numerous POPF risk calculators but no agreed benchmark, a key component of meaningful audit. We compared observed versus predicted POPF for six risk adjusted POPF calculators, to ascertain how they differ and thus contribute to discussion around benchmarking. METHODS: This was a retrospective single-arm cohort study at the Royal Melbourne Hospital of patients who underwent pancreaticoduodenectomy 1 November 2015 to 31 December 2021 with a primary outcome of a clinically relevant POPF. Cumulative sum (CUSUM) plots of observed versus predicted rate of POPF for sequential patients were constructed for six risk adjusted POPF calculators - Birmingham, updated Birmingham, fistula risk score (FRS), modified FRS (m-FRS), alternative FRS (a-FRS), and updated alternative FRS (ua-FRS). RESULTS: The study included 77 patients. The actual rate of clinically relevant POPF was 14.3%. FRS calculated an excess of 1.3 POPF per 100 cases. All other calculators demonstrated prevention of POPF per 100 cases: Birmingham 3.4, updated Birmingham 14.0, m-FRS 0.3, a-FRS 1.2, ua-FRS 19.7. CONCLUSION: The observed versus predicted rate of POPF was near zero for all risk calculators except ua-FRS and updated Birmingham, which predicted a higher POPF than observed (19.7, 14.0, respectively). These results indicate that, excepting ua-FRS and updated Birmingham, these calculators yield comparable results. Benchmarks for POPF should prescribe which risk calculators are used, and ideally a unified standard between centres should be the goal to provide consistency in outcome reporting and robust audit processes.