Bridging the Gap: Can Large Language Models Match Human Expertise in Writing Neurosurgical Operative Notes?

ABSTRACT

BACKGROUND: Proper documentation is essential for patient care. The popularity of artificial intelligence (AI) offers the potential for improvements in neurosurgical note-writing. The study aimed to assess how AI can optimize documentation in neurosurgical procedures. METHODS: Thirty-six notes were included. All identifiable data were removed. Essential information, such as perioperative data and diagnosis, was sourced from these notes. ChatGPT 4.0 was trained to draft notes from surgical vignettes using each surgeon's note template. One hundred forty-four surveys, with a surgeon or AI note, were shared with three surgeons to evaluate accuracy, content, and organization using a five-point scale. Accuracy was the factual correctness. Content was the comprehensiveness. Organization was the arrangement of the note. Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) scores quantified each note's readability. RESULTS: The mean AI accuracy (4.44) was not different from the mean surgeon accuracy (4.33, p = 0.512). The mean AI content (3.73) was lower than the mean surgeon content (4.42, p < 0.001). The mean AI organization (4.54) was greater than the mean surgeon organization (4.24, p = 0.064). The mean AI note's FKGL (13.13) was greater than the mean surgeon FKGL (9.99, p <0.001). The mean AI FRE (21.42) was lower than the mean surgeon FRE (41.70, p <0.001). CONCLUSION: AI notes were on par with surgeon notes in accuracy and organization, but lacked in content. Additionally, AI notes utilized language at an advanced reading level. These findings underscore the potential for ChatGPT to enhance the efficiency of neurosurgery documentation.