Evaluating the Efficacy of Large Language Models in CPT Coding for Craniofacial Surgery: A Comparative Analysis.

BACKGROUND: The advent of Large Language Models (LLMs) like ChatGPT has introduced significant advancements in various surgical disciplines. These developments have led to an increased interest in the utilization of LLMs for Current Procedural Terminology (CPT) coding in surgery. With CPT coding being a complex and time-consuming process, often exacerbated by the scarcity of professional coders, there is a pressing need for innovative solutions to enhance coding efficiency and accuracy. METHODS: This observational study evaluated the effectiveness of 5 publicly available large language models-Perplexity.AI, Bard, BingAI, ChatGPT 3.5, and ChatGPT 4.0-in accurately identifying CPT codes for craniofacial procedures. A consistent query format was employed to test each model, ensuring the inclusion of detailed procedure components where necessary. The responses were classified as correct, partially correct, or incorrect based on their alignment with established CPT coding for the specified procedures. RESULTS: The results indicate that while there is no overall significant association between the type of AI model and the correctness of CPT code identification, there are notable differences in performance for simple and complex CPT codes among the models. Specifically, ChatGPT 4.0 showed higher accuracy for complex codes, whereas Perplexity.AI and Bard were more consistent with simple codes. DISCUSSION: The use of AI chatbots for CPT coding in craniofacial surgery presents a promising avenue for reducing the administrative burden and associated costs of manual coding. Despite the lower accuracy rates compared with specialized, trained algorithms, the accessibility and minimal training requirements of the AI chatbots make them attractive alternatives. The study also suggests that priming AI models with operative notes may enhance their accuracy, offering a resource-efficient strategy for improving CPT coding in clinical practice. CONCLUSIONS: This study highlights the feasibility and potential benefits of integrating LLMs into the CPT coding process for craniofacial surgery. The findings advocate for further refinement and training of AI models to improve their accuracy and practicality, suggesting a future where AI-assisted coding could become a standard component of surgical workflows, aligning with the ongoing digital transformation in health care.

What Is the Value Proposition of Evidence-Based Guidelines? An Application of the Operative Value Index for Lumbar Fusions.

BACKGROUND: Elective lumbar fusions have received criticism for inappropriate utilization. Here, we use a novel Operative Value Index (OVI) to assess whether "indicated," evidence-based lumbar fusions are associated with increased value (outcomes per dollar spent). METHODS: This study is a retrospective analysis of a prospective observational cohort of 294 patients undergoing elective lumbar fusions at a single large academic institution. All patients were preoperatively evaluated by a panel of neurosurgeons for concordance with evidence-based medicine (EBM), determined through guidelines from the North American Spine Society. Oswestry Disability Index (ODI) scores were collected for all patients both preoperatively and at 6-months postoperatively. Time-driven activity-based costing was employed to determine both direct and indirect intraoperative costs. The OVI was defined as the percent improvement in ODI per $1000 spent intraoperatively. Generalized linear mixed model regression, adjusting for confounders, was performed to assess whether EBM-concordant surgeries were associated with higher OVI. RESULTS: Of 294 elective lumbar fusions, 92.9% (n = 273) were EBM-concordant. The average total cost of an EBM-concordant lumbar fusion was $17,932 (supplies: $13,020; personnel: $4314), compared to $20,616 (supplies: $15,467; personnel: $4758) for an EBM-discordant fusion. Average OVI was 2.27 for a concordant fusion, compared to 0.11 for a discordant fusion. Generalized linear mixed model analysis revealed that EBM-concordant cases were associated with significantly higher OVI (ß-coefficient 2.0, P < 0.001). CONCLUSIONS: EBM-concordant fusions were associated with 2% greater improvement in ODI scores from baseline for every $1000 spent intraoperatively. Systematic methods for increasing guideline adherence for lumbar fusions could therefore improve value at scale.

The One Health Consortium: Design of a Phase I Clinical Trial to Evaluate M032, a Genetically Engineered HSV-1 Expressing IL-12, in Combination With a Checkpoint Inhibitor in Canine Patients With Sporadic High Grade Gliomas.

As the most common and deadly of primary brain tumors, malignant gliomas have earned their place within one of the most multifaceted and heavily-funded realms of medical research. Numerous avenues of pre-clinical investigation continue to provide valuable insight, but modeling the complex evolution and behavior of these tumors within a host under simulated circumstances may pose challenges to extrapolation of data. Remarkably, certain breeds of pet dogs spontaneously and sporadically develop high grade gliomas that follow similar incidence, treatment, and outcome patterns as their human glioma counterparts. The most malignant of these tumors have been refractory to limited treatment options despite aggressive treatment; outcomes are dismal with median survivals of just over 1 year in humans and 2 months in dogs. Novel treatments are greatly needed and combination therapies appear to hold promise. This clinical protocol, a dose-escalating phase I study in dogs with sporadic malignant glioma, represents a first in comparative oncology and combination immunotherapy. The trial will evaluate M032, an Interleukin-12 expressing Herpes Simplex virus, alone and combined with a checkpoint inhibitor, Indoximod. Extensive pre-clinical work has demonstrated safety of intracranial M032 administration in mice and non-human primates. M032 is currently being tested in humans with high-grade malignant gliomas. Thus, in a novel fashion, both canine and human trials will proceed concurrently allowing a direct "head-to-head" comparison of safety and efficacy. We expect this viral oncolytic therapy to be as safe as it is in human patients and M032 to (a) infect and kill glioma cells, producing a virus and tumor cell antigen-rich debris field; (b) provide an adjuvant effect due to liberation of viral DNA, which is rich in unmethylated CpG sequences that "toggle" TLR-9 receptors; and (c) express IL-12 locally, stimulating induction of TH1 lymphocytes. The resultant immune-mediated anti-viral responses should, through cross-epitope spreading, translate into a strong response to tumor antigens. The ability to compare human and dog responses in real time affords the most stringent test of suitability of the dog as an informative model of human brain tumors. Subsequent studies will allow canine trials to properly inform the design of human trials.

Safety of Running Two Rooms: A Systematic Review and Meta-Analysis of Overlapping Neurosurgical Procedures.

OBJECTIVE: Overlapping surgery, a long-standing practice within academic neurosurgery centers nationwide, has recently come under scrutiny from the government and media as potentially harmful to patients. Therefore, the objective of this systematic review and meta-analysis is to determine the safety of overlapping neurosurgical procedures. METHODS: The authors performed a systematic review and meta-analysis in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. A review of PubMed and Medline databases was undertaken with the search phrase "overlapping surgery AND neurosurgery AND outcomes." Data regarding patient demographics, type of neurosurgical procedure, outcomes, and complications were extracted from each study. The principle summary measure was odds ratio (OR) of the association of overlapping versus non-overlapping surgery with outcomes. RESULTS: The literature search yielded a total of 36 studies, of which 5 studies met inclusion criteria and were included in this study. These studies included a total of 25,764 patients undergoing neurosurgical procedures. Overlapping surgery was associated with an increased likelihood of being discharged home (OR, 1.32; 95% confidence interval [CI], 1.20-1.44; P < 0.001) and a reduced 30-day unexpected return to the operating room (OR, 0.79; 95% CI, 0.72-0.87; P < 0.001). Overlapping surgery did not significantly affect OR of length of surgery, 30-day mortality, or 30-day readmission. CONCLUSIONS: Overlapping neurosurgical procedures were not associated with worse patient outcomes. In addition, prospective studies are needed to assess the safety overlapping procedures.