Validation of a surgical simulator and establishment of quantitative performance thresholds-RealSpine simulation system for open lumbar decompressions.

BACKGROUND CONTEXT: The majority of surgical training is conducted in real-world operations. High-fidelity surgical simulators may provide a safer environment for surgical training. However, the extent that it reflects real-world operations and surgical ability is often poorly characterized. PURPOSE: (1) Assess the validity and fidelity of a surgical simulator; (2) Examine the quantitative relationship between simulation performance and markers of real-world ability; (3) Establish thresholds for surgical expertise, and estimate their external validity and accuracy. STUDY DESIGN/SETTING: A cohort study of surgeons at a British neurosurgical center. STUDY SAMPLE: Ten early-career "novice" surgeons and 8 board-certified "expert" neurosurgeons. OUTCOMES MEASURES: (1) Face and content validity, and visual and haptic fidelity; (2) Construct validity; (3) Predictive and discriminative utility of quantitative performance thresholds. METHODS: Participants performed unilateral lumbar decompressions on high-fidelity spinal simulators that replicate the bony and soft tissue anatomy along with physiological processes such as bleeding and CSF leaks. Operating times, measured from first surgical action to either self-perceived satisfactory decompression or the end of allocated time, were recorded. The performance was also assessed independently by 2 blinded spinal subspecialist neurosurgeons using OSATS, a validated surgical assessment tool that utilizes 5-point scales on a variety of technical domains to grade the overall technical proficiency. Validity and fidelity were assessed by expert neurosurgeons using quantitative questionnaires. Construct validity was assessed by ordinal regression of simulation performance against real-world surgical grade and portfolio. Thresholds of expert status by simulation performance was established, and their predictive and discriminative utility assessed by crossvalidation accuracy and AUC-ROC. RESULTS: Operating time and expert assessments of simulation performance (OSATS) were strong and significant prdictors of surrogate markers of real-world surgical ability. The thresholds for expert status were operating time of 15 minutes and modified OSATS score of 15/20. These thresholds predicted expert status with 84.2% and 71.4% accuracy respectively. Strong discriminative ability was demonstrated by AUC-ROC of 0.95 and 0.83 respectively. All expert surgeons agreed that RealSpine simulators demonstrate high face validity, and high visual and haptic fidelity, with overall scores showing statistically significant agreement on these items (all scores at least 4/5, p<.0001). There was less consensus on content validity, but with still significant overall agreement (average score: 3.75/5, p=.023). CONCLUSIONS: Real-world surgical ability and experience can be accurately predicted by defining objective quantitative thresholds on high-fidelity simulations. The thresholds established here, along with other data presented in this paper, may inform objectives and standards to be established in a spinal surgical training curriculum.

Mapping the Clinical Pathway for Patients Undergoing Vestibular Schwannoma Resection.

BACKGROUND: The introduction of the electronic health record (EHR) has improved the collection and storage of patient information, enhancing clinical communication and academic research. However, EHRs are limited by data quality and the time-consuming task of manual data extraction. This study aimed to use process mapping to help identify critical data entry points within the clinical pathway for patients with vestibular schwannoma (VS) ideal for structured data entry and automated data collection to improve patient care and research. METHODS: A 2-stage methodology was used at a neurosurgical unit. Process maps were developed using semi-structured interviews with stakeholders in the management of VS resection. Process maps were then retrospectively validated against EHRs for patients admitted between August 2019 and December 2021, establishing critical data entry points. RESULTS: In the process map development, 20 stakeholders were interviewed. Process maps were validated against EHRs of 36 patients admitted for VS resection. Operative notes, surgical inpatient reviews (including ward rounds), and discharge summaries were available for all patients, representing critical data entry points. Areas for documentation improvement were in the preoperative clinics (30/36; 83.3%), preoperative skull base multidisciplinary team (32/36; 88.9%), postoperative follow-up clinics (32/36; 88.9%), and postoperative skull base multidisciplinary team meeting (29/36; 80.6%). CONCLUSIONS: This is a first use to our knowledge of a 2-stage methodology for process mapping the clinical pathway for patients undergoing VS resection. We identified critical data entry points that can be targeted for structured data entry and for automated data collection tools, positively impacting patient care and research.

Early evaluation of a natural language processing tool to improve access to educational resources for surgical patients.

PURPOSE: Accessible patient information sources are vital in educating patients about the benefits and risks of spinal surgery, which is crucial for obtaining informed consent. We aim to assess the effectiveness of a natural language processing (NLP) pipeline in recognizing surgical procedures from clinic letters and linking this with educational resources. METHODS: Retrospective examination of letters from patients seeking surgery for degenerative spinal disease at a single neurosurgical center. We utilized MedCAT, a named entity recognition and linking NLP, integrated into the electronic health record (EHR), which extracts concepts and links them to systematized nomenclature of medicine-clinical terms (SNOMED-CT). Investigators reviewed clinic letters, identifying words or phrases that described or identified operations and recording the SNOMED-CT terms as ground truth. This was compared to SNOMED-CT terms identified by the model, untrained on our dataset. A pipeline linking clinic letters to patient-specific educational resources was established, and precision, recall, and F1 scores were calculated. RESULTS: Across 199 letters the model identified 582 surgical procedures, and the overall pipeline after adding rules a total of 784 procedures (precision = 0.94, recall = 0.86, F1 = 0.91). Across 187 letters with identified SNOMED-CT terms the integrated pipeline linking education resources directly to the EHR was successful in 157 (78%) patients (precision = 0.99, recall = 0.87, F1 = 0.92). CONCLUSIONS: NLP accurately identifies surgical procedures in pre-operative clinic letters within an untrained subspecialty. Performance varies among letter authors and depends on the language used by clinicians. The identified procedures can be linked to patient education resources, potentially improving patients' understanding of surgical procedures.

Mapping patient education encounters in elective surgery: a cohort study and cross-sectional survey.

OBJECTIVE: Develop a process map of when patients learn about their proposed surgery and what resources patients use to educate themselves. DESIGN: A mixed methods design, combining semistructured stakeholder interviews, quantitative validation using electronic healthcare records (EHR) in a retrospective cohort and a cross-sectional patient survey. SETTING: A single surgical centre in the UK. PARTICIPANTS: Fourteen members of the spinal multidisciplinary team were interviewed to develop the process map.This process map was validated using the EHR of 50 patients undergoing elective spine surgery between January and June 2022. Postprocedure, feedback was gathered from 25 patient surveys to identify which resources they used to learn about their spinal procedure. Patients below the age of 18 or who received emergency surgery were excluded. INTERVENTIONS: Elective spine surgery and patient questionnaires given postoperatively either on the ward or in follow-up clinic. PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcome was the percentage of the study cohort that was present at encounters on the process map. Key timepoints were defined if >80% of patients were present. The secondary outcome was the percentage of the study cohort that used educational resources listed in the patient questionnaire. RESULTS: There were 342 encounters which occurred across the cohort, with 16 discrete event categories identified. The initial surgical clinic (88%), anaesthetic preoperative assessment (96%) and admission for surgery (100%) were identified as key timepoints. Surveys identified that patients most used verbal information from their surgeon (100%) followed by written information from their surgeon (52%) and the internet (40%) to learn about their surgery. CONCLUSIONS: Process mapping is an effective method of illustrating the patient pathway. The initial surgical clinic, anaesthetic preoperative assessment and surgical admission are key timepoints where patients receive information. This has future implications for guiding patient education interventions to focus at key timepoints.

Image-guidance in endoscopic pituitary surgery: an in-silico study of errors involved in tracker-based techniques.

Background: Endoscopic endonasal surgery is an established minimally invasive technique for resecting pituitary adenomas. However, understanding orientation and identifying critical neurovascular structures in this anatomically dense region can be challenging. In clinical practice, commercial navigation systems use a tracked pointer for guidance. Augmented Reality (AR) is an emerging technology used for surgical guidance. It can be tracker based or vision based, but neither is widely used in pituitary surgery. Methods: This pre-clinical study aims to assess the accuracy of tracker-based navigation systems, including those that allow for AR. Two setups were used to conduct simulations: (1) the standard pointer setup, tracked by an infrared camera; and (2) the endoscope setup that allows for AR, using reflective markers on the end of the endoscope, tracked by infrared cameras. The error sources were estimated by calculating the Euclidean distance between a point's true location and the point's location after passing it through the noisy system. A phantom study was then conducted to verify the in-silico simulation results and show a working example of image-based navigation errors in current methodologies. Results: The errors of the tracked pointer and tracked endoscope simulations were 1.7 and 2.5 mm respectively. The phantom study showed errors of 2.14 and 3.21 mm for the tracked pointer and tracked endoscope setups respectively. Discussion: In pituitary surgery, precise neighboring structure identification is crucial for success. However, our simulations reveal that the errors of tracked approaches were too large to meet the fine error margins required for pituitary surgery. In order to achieve the required accuracy, we would need much more accurate tracking, better calibration and improved registration techniques.

High fidelity simulation of the endoscopic transsphenoidal approach: Validation of the UpSurgeOn TNS Box.

Objective: Endoscopic endonasal transsphenoidal surgery is an established technique for the resection of sellar and suprasellar lesions. The approach is technically challenging and has a steep learning curve. Simulation is a growing training tool, allowing the acquisition of technical skills pre-clinically and potentially resulting in a shorter clinical learning curve. We sought validation of the UpSurgeOn Transsphenoidal (TNS) Box for the endoscopic endonasal transsphenoidal approach to the pituitary fossa. Methods: Novice, intermediate and expert neurosurgeons were recruited from multiple centres. Participants were asked to perform a sphenoidotomy using the TNS model. Face and content validity were evaluated using a post-task questionnaire. Construct validity was assessed through post-hoc blinded scoring of operative videos using a Modified Objective Structured Assessment of Technical Skills (mOSAT) and a Task-Specific Technical Skill scoring system. Results: Fifteen participants were recruited of which n = 10 (66.6%) were novices and n = 5 (33.3%) were intermediate and expert neurosurgeons. Three intermediate and experts (60%) agreed that the model was realistic. All intermediate and experts (n = 5) strongly agreed or agreed that the TNS model was useful for teaching the endonasal transsphenoidal approach to the pituitary fossa. The consensus-derived mOSAT score was 16/30 (IQR 14-16.75) for novices and 29/30 (IQR 27-29) for intermediate and experts (p < 0.001, Mann-Whitney U). The median Task-Specific Technical Skill score was 10/20 (IQR 8.25-13) for novices and 18/20 (IQR 17.75-19) for intermediate and experts (p < 0.001, Mann-Whitney U). Interrater reliability was 0.949 (CI 0.983-0.853) for OSATS and 0.945 (CI 0.981-0.842) for Task-Specific Technical Skills. Suggested improvements for the model included the addition of neuro-vascular anatomy and arachnoid mater to simulate bleeding vessels and CSF leak, respectively, as well as improvement in materials to reproduce the consistency closer to that of human tissue and bone. Conclusion: The TNS Box simulation model has demonstrated face, content, and construct validity as a simulator for the endoscopic endonasal transsphenoidal approach. With the steep learning curve associated with endoscopic approaches, this simulation model has the potential as a valuable training tool in neurosurgery with further improvements including advancing simulation materials, dynamic models (e.g., with blood flow) and synergy with complementary technologies (e.g., artificial intelligence and augmented reality).