RESUMO
PURPOSE: Lumbar discectomy is among the most common spine procedures in the US, with 300,000 procedures performed each year. Like other surgical procedures, this procedure is not excluded from potential complications. This paper presents a video annotation methodology for microdiscectomy including the development of a surgical workflow. In future work, this methodology could be combined with computer vision and machine learning models to predict potential adverse events. These systems would monitor the intraoperative activities and possibly anticipate the outcomes. METHODS: A necessary step in supervised machine learning methods is video annotation, which involves labeling objects frame-by-frame to make them recognizable for machine learning applications. Microdiscectomy video recordings of spine surgeries were collected from a multi-center research collaborative. These videos were anonymized and stored in a cloud-based platform. Videos were uploaded to an online annotation platform. An annotation framework was developed based on literature review and surgical observations to ensure proper understanding of the instruments, anatomy, and steps. RESULTS: An annotated video of microdiscectomy was produced by a single surgeon. Multiple iterations allowed for the creation of an annotated video complete with labeled surgical tools, anatomy, and phases. In addition, a workflow was developed for the training of novice annotators, which provides information about the annotation software to assist in the production of standardized annotations. CONCLUSIONS: A standardized workflow for managing surgical video data is essential for surgical video annotation and machine learning applications. We developed a standard workflow for annotating surgical videos for microdiscectomy that may facilitate the quantitative analysis of videos using supervised machine learning applications. Future work will demonstrate the clinical relevance and impact of this workflow by developing process modeling and outcome predictors.
RESUMO
OBJECTIVE: To explore optimal feedback methodologies to enhance trainee skill acquisition in simulated surgical bimanual skills learning during brain tumor resections. HYPOTHESES: (1) Providing feedback results in better learning outcomes in teaching surgical technical skill when compared to practice alone with no tailored performance feedback. (2) Providing more visual and visuospatial feedback results in better learning outcomes when compared to providing numerical feedback. DESIGN: A prospective 4-parallel-arm randomized controlled trial. SETTING: Neurosurgical Simulation and Artificial Intelligence Learning Centre, McGill University, Canada. PARTICIPANTS: Medical students (nâ¯=â¯120) from 4 Quebec medical schools. RESULTS: Participants completed a virtually simulated tumor resection task 5 times while receiving 1 of 4 feedback based on their group allocation: (1) practice-alone without feedback, (2) numerical feedback, (3) visual feedback, and (4) visuospatial feedback. Outcome measures were participants' scores on 14-performance metrics and the number of expert benchmarks achieved during each task. There were no significant differences in the first task which determined baseline performance. A statistically significant interaction between feedback allocation and task repetition was found on the number of benchmarks achieved, F (10.558, 408.257)=3.220, p < 0.001. Participants in all feedback groups significantly improved their performance compared to baseline. The visual feedback group achieved significantly higher number of benchmarks than the practice-alone group by the third repetition of the task, pâ¯=â¯0.005, 95%CI [0.42 3.25]. Visual feedback and visuospatial feedback improved performance significantly by the second repetition of the task, pâ¯=â¯0.016, 95%CI [0.19 2.71] and pâ¯=â¯0.003, 95%CI [0.4 2.57], respectively. CONCLUSION: Simulations with autonomous visual computer assistance may be effective pedagogical tools in teaching bimanual operative skills via visual and visuospatial feedback information delivery.
Assuntos
Inteligência Artificial , Treinamento por Simulação , Humanos , Retroalimentação , Estudos Prospectivos , Treinamento por Simulação/métodos , Simulação por Computador , Competência ClínicaRESUMO
BACKGROUND AND OBJECTIVES: Subpial corticectomy involving complete lesion resection while preserving pial membranes and avoiding injury to adjacent normal tissues is an essential bimanual task necessary for neurosurgical trainees to master. We sought to develop an ex vivo calf brain corticectomy simulation model with continuous assessment of surgical instrument movement during the simulation. A case series study of skilled participants was performed to assess face and content validity to gain insights into the utility of this training platform, along with determining if skilled and less skilled participants had statistical differences in validity assessment. METHODS: An ex vivo calf brain simulation model was developed in which trainees performed a subpial corticectomy of three defined areas. A case series study assessed face and content validity of the model using 7-point Likert scale questionnaires. RESULTS: Twelve skilled and 11 less skilled participants were included in this investigation. Overall median scores of 6.0 (range 4.0-6.0) for face validity and 6.0 (range 3.5-7.0) for content validity were determined on the 7-point Likert scale, with no statistical differences between skilled and less skilled groups identified. CONCLUSION: A novel ex vivo calf brain simulator was developed to replicate the subpial resection procedure and demonstrated face and content validity.
RESUMO
Trainees develop surgical technical skills by learning from experts who provide context for successful task completion, identify potential risks, and guide correct instrument handling. This expert-guided training faces significant limitations in objectively assessing skills in real-time and tracking learning. It is unknown whether AI systems can effectively replicate nuanced real-time feedback, risk identification, and guidance in mastering surgical technical skills that expert instructors offer. This randomized controlled trial compared real-time AI feedback to in-person expert instruction. Ninety-seven medical trainees completed a 90-min simulation training with five practice tumor resections followed by a realistic brain tumor resection. They were randomly assigned into 1-real-time AI feedback, 2-in-person expert instruction, and 3-no real-time feedback. Performance was assessed using a composite-score and Objective Structured Assessment of Technical Skills rating, rated by blinded experts. Training with real-time AI feedback (n = 33) resulted in significantly better performance outcomes compared to no real-time feedback (n = 32) and in-person instruction (n = 32), .266, [95% CI .107 .425], p < .001; .332, [95% CI .173 .491], p = .005, respectively. Learning from AI resulted in similar OSATS ratings (4.30 vs 4.11, p = 1) compared to in-person training with expert instruction. Intelligent systems may refine the way operating skills are taught, providing tailored, quantifiable feedback and actionable instructions in real-time.
Assuntos
Inteligência Artificial , Competência Clínica , Humanos , Feminino , Masculino , Adulto , Treinamento por Simulação/métodosRESUMO
Virtual reality surgical simulators have facilitated surgical education by providing a safe training environment. Electroencephalography (EEG) has been employed to assess neuroelectric activity during surgical performance. Machine learning (ML) has been applied to analyze EEG data split into frequency bands. Although EEG is widely used in fields requiring expert performance, it has yet been used to classify surgical expertise. Thus, the goals of this study were to (a) develop an ML model to accurately differentiate skilled and less-skilled performance using EEG data recorded during a simulated surgery, (b) explore the relative importance of each EEG bandwidth to expertise, and (c) analyze differences in EEG band powers between skilled and less-skilled individuals. We hypothesized that EEG recordings during a virtual reality surgery task would accurately predict the expertise level of the participant. Twenty-one participants performed three simulated brain tumor resection procedures on the NeuroVR™ platform (CAE Healthcare, Montreal, Canada) while EEG data was recorded. Participants were divided into 2 groups. The skilled group was composed of five neurosurgeons and five senior neurosurgical residents (PGY4-6), and the less-skilled group was composed of six junior residents (PGY1-3) and five medical students. A total of 13 metrics from EEG frequency bands and ratios (e.g., alpha, theta/beta ratio) were generated. Seven ML model types were trained using EEG activity to differentiate between skilled and less-skilled groups. The artificial neural network achieved the highest testing accuracy of 100% (AUROCâ¯=â¯1.0). Model interpretation via Shapley analysis identified low alpha (8-10â¯Hz) as the most important metric for classifying expertise. Skilled surgeons displayed higher (pâ¯=â¯0.044) low-alpha than the less-skilled group. Furthermore, skilled surgeons displayed significantly lower TBR (pâ¯=â¯0.048) and significantly higher beta (13-30â¯Hz, pâ¯=â¯0.049), beta 1 (15-18â¯Hz, pâ¯=â¯0.014), and beta 2 (19-22â¯Hz, pâ¯=â¯0.015), thus establishing these metrics as important markers of expertise. ACGME CORE COMPETENCIES: Practice-Based Learning and Improvement.
Assuntos
Inteligência Artificial , Realidade Virtual , Humanos , Aprendizado de Máquina , Eletroencefalografia , Redes Neurais de ComputaçãoRESUMO
Importance: To better elucidate the role of artificial intelligence (AI) in surgical skills training requires investigations in the potential existence of a hidden curriculum. Objective: To assess the pedagogical value of AI-selected technical competencies and their extended effects in surgical simulation training. Design, Setting, and Participants: This cohort study was a follow-up of a randomized clinical trial conducted at the Neurosurgical Simulation and Artificial Intelligence Learning Centre at the Montreal Neurological Institute, McGill University, Montreal, Canada. Surgical performance metrics of medical students exposed to an AI-enhanced training curriculum were compared with a control group of participants who received no feedback and with expert benchmarks. Cross-sectional data were collected from January to April 2021 from medical students and from March 2015 to May 2016 from experts. This follow-up secondary analysis was conducted from June to September 2022. Participants included medical students (undergraduate year 0-2) in the intervention cohorts and neurosurgeons to establish expertise benchmarks. Exposure: Performance assessment and personalized feedback by an intelligent tutor on 4 AI-selected learning objectives during simulation training. Main Outcomes and Measures: Outcomes of interest were unintended performance outcomes, measured by significant within-participant difference from baseline in 270 performance metrics in the intervention cohort that was not observed in the control cohort. Results: A total of 46 medical students (median [range] age, 22 [18-27] years; 27 [59%] women) and 14 surgeons (median [range] age, 45 [35-59] years; 14 [100%] men) were included in this study, and no participant was lost to follow-up. Feedback on 4 AI-selected technical competencies was associated with additional performance change in 32 metrics over the entire procedure and 20 metrics during tumor removal that was not observed in the control group. Participants exposed to the AI-enhanced curriculum demonstrated significant improvement in safety metrics, such as reducing the rate of healthy tissue removal (mean difference, -7.05 × 10-5 [95% CI, -1.09 × 10-4 to -3.14 × 10-5] mm3 per 20 ms; P < .001) and maintaining a focused bimanual control of the operative field (mean difference in maximum instrument divergence, -4.99 [95% CI, -8.48 to -1.49] mm, P = .006) compared with the control group. However, negative unintended effects were also observed. These included a significantly lower velocity and acceleration in the dominant hand (velocity: mean difference, -0.13 [95% CI, -0.17 to -0.09] mm per 20 ms; P < .001; acceleration: mean difference, -2.25 × 10-2 [95% CI, -3.20 × 10-2 to -1.31 × 10-2] mm per 20 ms2; P < .001) and a significant reduction in the rate of tumor removal (mean difference, -4.85 × 10-5 [95% CI, -7.22 × 10-5 to -2.48 × 10-5] mm3 per 20 ms; P < .001) compared with control. These unintended outcomes diverged students' movement and efficiency performance metrics away from the expertise benchmarks. Conclusions and Relevance: In this cohort study of medical students, an AI-enhanced curriculum for bimanual surgical skills resulted in unintended changes that improved performance in safety but negatively affected some efficiency metrics. Incorporating AI in course design requires ongoing assessment to maintain transparency and foster evidence-based learning objectives.
Assuntos
Neoplasias , Treinamento por Simulação , Masculino , Feminino , Humanos , Adulto Jovem , Adulto , Pessoa de Meia-Idade , Inteligência Artificial , Estudos de Coortes , Estudos Transversais , CurrículoRESUMO
BACKGROUND AND OBJECTIVES: Anterior cervical discectomy and fusion (ACDF) is among the most common spine procedures. The Sim-Ortho virtual reality simulator platform contains a validated ACDF simulated task for performance assessment. This study aims to develop a methodology to extract three-dimensional data and reconstruct and quantitate specific simulated disc tissues to generate novel metrics to analyze performance metrics of skilled and less skilled participants. METHODS: We used open-source platforms to develop a methodology to extract three-dimensional information from ACDF simulation data. Metrics generated included, efficiency index, disc volumes removed from defined regions, and rate of tissue removal from superficial, central, and deep disc regions. A pilot study was performed to assess the utility of this methodology to assess expertise during the ACDF simulated procedure. RESULTS: The system outlined, extracts data allowing the development of a methodology which accurately reconstructs and quantitates 3-dimensional disc volumes. In the pilot study, data sets from 27 participants, divided into postresident, resident, and medical student groups, allowed assessment of multiple novel metrics, including efficiency index (surgical time spent in actively removing disc), where the postresident group spent 61.8% of their time compared with 53% and 30.2% for the resident and medical student groups, respectively ( P = .01). During the annulotomy component, the postresident group removed 47.4% more disc than the resident groups and 102% more than the medical student groups ( P = .03). CONCLUSION: The methodology developed in this study generates novel surgical procedural metrics from 3-dimensional data generated by virtual reality simulators and can be used to assess surgical performance.
Assuntos
Fusão Vertebral , Realidade Virtual , Humanos , Projetos Piloto , Vértebras Cervicais/cirurgia , Fusão Vertebral/métodos , Discotomia/métodosRESUMO
BACKGROUND: The methodology of assessment and training of surgical skills is evolving to deal with the emergence of competency-based training. Artificial neural networks (ANNs), a branch of artificial intelligence, can use newly generated metrics not only for assessment performance but also to quantitate individual metric importance and provide new insights into surgical expertise. OBJECTIVE: To outline the educational utility of using an ANN in the assessment and quantitation of surgical expertise. A virtual reality vertebral osteophyte removal during a simulated surgical spine procedure is used as a model to outline this methodology. METHODS: Twenty-one participants performed a simulated anterior cervical diskectomy and fusion on the Sim-Ortho virtual reality simulator. Participants were divided into 3 groups, including 9 postresidents, 5 senior residents, and 7 junior residents. Data were retrieved from the osteophyte removal component of the scenario, which involved using a simulated burr. The data were manipulated to initially generate 83 performance metrics spanning 3 categories (safety, efficiency, and motion) of which only the most relevant metrics were used to train and test the ANN. RESULTS: The ANN model was trained on 6 safety metrics to a testing accuracy of 83.3%. The contributions of these performance metrics to expertise were revealed through connection weight products and outlined 2 identifiable learning patterns of technical skills. CONCLUSION: This study outlines the potential utility of ANNs which allows a deeper understanding of the composites of surgical expertise and may contribute to the paradigm shift toward competency-based surgical training.
Assuntos
Osteófito , Realidade Virtual , Inteligência Artificial , Competência Clínica , Humanos , Redes Neurais de ComputaçãoRESUMO
OBJECTIVE: Understanding the variation of learning curves of experts and trainees for a given surgical procedure is important in implementing formative learning paradigms to accelerate mastery. The study objectives were to use artificial intelligence (AI)-derived metrics to determine the learning curves of participants in 4 groups with different expertise levels who performed a series of identical virtual reality (VR) subpial resection tasks and to identify learning curve differences among the 4 groups. METHODS: A total of 50 individuals participated, 14 neurosurgeons, 4 neurosurgical fellows and 10 senior residents (seniors), 10 junior residents (juniors), and 12 medical students. All participants performed 5 repetitions of a subpial tumor resection on the NeuroVR (CAE Healthcare) platform, and 6 a priori-derived metrics selected using the K-nearest neighbors machine learning algorithm were used to assess participant learning curves. Group learning curves were plotted over the 5 trials for each metric. A mixed, repeated-measures ANOVA was performed between the first and fifth trial. For significant interactions (p < 0.05), post hoc Tukey's HSD analysis was conducted to determine the location of the significance. RESULTS: Overall, 5 of the 6 metrics assessed had a significant interaction (p < 0.05). The 4 groups, neurosurgeons, seniors, juniors, and medical students, showed an improvement between the first and fifth trial on at least one of the 6 metrics evaluated. CONCLUSIONS: Learning curves generated using AI-derived metrics provided novel insights into technical skill acquisition, based on expertise level, during repeated VR-simulated subpial tumor resections, which will allow educators to develop more focused formative educational paradigms for neurosurgical trainees.
RESUMO
BACKGROUND: Virtual reality surgical simulators provide detailed psychomotor performance data, allowing qualitative and quantitative assessment of hand function. The nondominant hand plays an essential role in neurosurgery in exposing the operative area, assisting the dominant hand to optimize task execution, and hemostasis. Outlining expert-level nondominant hand skills may be critical to understand surgical expertise and aid learner training. OBJECTIVE: To (1) provide validity for the simulated bimanual subpial tumor resection task and (2) to use this simulation in qualitative and quantitative evaluation of nondominant hand skills for bipolar forceps utilization. METHODS: In this case series study, 45 right-handed participants performed a simulated subpial tumor resection using simulated bipolar forceps in the nondominant hand for assisting the surgery and hemostasis. A 10-item questionnaire was used to assess task validity. The nondominant hand skills across 4 expertise levels (neurosurgeons, senior trainees, junior trainees, and medical students) were analyzed by 2 visual models and performance metrics. RESULTS: Neurosurgeon median (range) overall satisfaction with the simulated scenario was 4.0/5.0 (2.0-5.0). The visual models demonstrated a decrease in high force application areas on pial surface with increased expertise level. Bipolar-pia mater interactions were more focused around the tumoral region for neurosurgeons and senior trainees. These groups spent more time using the bipolar while interacting with pia. All groups spent significantly higher time in the left upper pial quadrant than other quadrants. CONCLUSION: This work introduces new approaches for the evaluation of nondominant hand skills which may help surgical trainees by providing both qualitative and quantitative feedback.
Assuntos
Neoplasias Encefálicas , Neurocirurgia , Treinamento por Simulação , Realidade Virtual , Neoplasias Encefálicas/cirurgia , Humanos , Neurocirurgiões , Neurocirurgia/educaçãoRESUMO
In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.
RESUMO
Importance: To better understand the emerging role of artificial intelligence (AI) in surgical training, efficacy of AI tutoring systems, such as the Virtual Operative Assistant (VOA), must be tested and compared with conventional approaches. Objective: To determine how VOA and remote expert instruction compare in learners' skill acquisition, affective, and cognitive outcomes during surgical simulation training. Design, Setting, and Participants: This instructor-blinded randomized clinical trial included medical students (undergraduate years 0-2) from 4 institutions in Canada during a single simulation training at McGill Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal, Canada. Cross-sectional data were collected from January to April 2021. Analysis was conducted based on intention-to-treat. Data were analyzed from April to June 2021. Interventions: The interventions included 5 feedback sessions, 5 minutes each, during a single 75-minute training, including 5 practice sessions followed by 1 realistic virtual reality brain tumor resection. The 3 intervention arms included 2 treatment groups, AI audiovisual metric-based feedback (VOA group) and synchronous verbal scripted debriefing and instruction from a remote expert (instructor group), and a control group that received no feedback. Main Outcomes and Measures: The coprimary outcomes were change in procedural performance, quantified as Expertise Score by a validated assessment algorithm (Intelligent Continuous Expertise Monitoring System [ICEMS]; range, -1.00 to 1.00) for each practice resection, and learning and retention, measured from performance in realistic resections by ICEMS and blinded Objective Structured Assessment of Technical Skills (OSATS; range 1-7). Secondary outcomes included strength of emotions before, during, and after the intervention and cognitive load after intervention, measured in self-reports. Results: A total of 70 medical students (41 [59%] women and 29 [41%] men; mean [SD] age, 21.8 [2.3] years) from 4 institutions were randomized, including 23 students in the VOA group, 24 students in the instructor group, and 23 students in the control group. All participants were included in the final analysis. ICEMS assessed 350 practice resections, and ICEMS and OSATS evaluated 70 realistic resections. VOA significantly improved practice Expertise Scores by 0.66 (95% CI, 0.55 to 0.77) points compared with the instructor group and by 0.65 (95% CI, 0.54 to 0.77) points compared with the control group (P < .001). Realistic Expertise Scores were significantly higher for the VOA group compared with instructor (mean difference, 0.53 [95% CI, 0.40 to 0.67] points; P < .001) and control (mean difference. 0.49 [95% CI, 0.34 to 0.61] points; P < .001) groups. Mean global OSATS ratings were not statistically significant among the VOA (4.63 [95% CI, 4.06 to 5.20] points), instructor (4.40 [95% CI, 3.88-4.91] points), and control (3.86 [95% CI, 3.44 to 4.27] points) groups. However, on the OSATS subscores, VOA significantly enhanced the mean OSATS overall subscore compared with the control group (mean difference, 1.04 [95% CI, 0.13 to 1.96] points; P = .02), whereas expert instruction significantly improved OSATS subscores for instrument handling vs control (mean difference, 1.18 [95% CI, 0.22 to 2.14]; P = .01). No significant differences in cognitive load, positive activating, and negative emotions were found. Conclusions and Relevance: In this randomized clinical trial, VOA feedback demonstrated superior performance outcome and skill transfer, with equivalent OSATS ratings and cognitive and emotional responses compared with remote expert instruction, indicating advantages for its use in simulation training. Trial Registration: ClinicalTrials.gov Identifier: NCT04700384.
Assuntos
Inteligência Artificial , Educação Médica/métodos , Cirurgia Geral/educação , Treinamento por Simulação , Estudantes de Medicina , Adulto , Canadá , Competência Clínica , Avaliação Educacional , Feminino , Humanos , Masculino , Realidade Virtual , Adulto JovemRESUMO
BACKGROUND: Ex vivo animal brain simulation models are being increasingly used for neurosurgical training because these models can replicate human brain conditions. The goal of the present report is to provide the neurosurgical community interested in using ex vivo animal brain simulation models with guidelines for comprehensively and rigorously conducting, documenting, and assessing this type of research. METHODS: In consultation with an interdisciplinary group of physicians and researchers involved in ex vivo models and a review of the literature on the best practices guidelines for simulation research, we developed the "ex vivo brain model to assess surgical expertise" (EVBMASE) checklist. The EVBMASE checklist provides a comprehensive quantitative framework for analyzing and reporting studies involving these models. We applied The EVBMASE checklist to the studies reported of ex vivo animal brain models to document how current ex vivo brain simulation models are used to train surgical expertise. RESULTS: The EVBMASE checklist includes defined subsections and a total score of 20, which can help investigators improve studies and provide readers with techniques to better assess the quality and any deficiencies of the research. We classified 18 published ex vivo brain models into modified (group 1) and nonmodified (group 2) models. The mean total EVBMASE score was 11 (55%) for group 1 and 4.8 (24.2%) for group 2, a statistically significant difference (P = 0.006) mainly attributed to differences in the simulation study design section (P = 0.003). CONCLUSIONS: The present findings should help contribute to more rigorous application, documentation, and assessment of ex vivo brain simulation research.
Assuntos
Encéfalo/cirurgia , Competência Clínica/normas , Modelos Anatômicos , Neurocirurgia/educação , Neurocirurgia/normas , Guias de Prática Clínica como Assunto/normas , Animais , Encéfalo/anatomia & histologia , Encéfalo/patologia , Encefalopatias/patologia , Encefalopatias/cirurgia , Bovinos , Lista de Checagem/normas , Humanos , Ovinos , SuínosRESUMO
BACKGROUND: Virtual reality surgical simulators are a safe and efficient technology for the assessment and training of surgical skills. Simulators allow trainees to improve specific surgical techniques in risk-free environments. Recently, machine learning has been coupled to simulators to classify performance. However, most studies fail to extract meaningful observations behind the classifications and the impact of specific surgical metrics on the performance. One benefit from integrating machine learning algorithms, such as Artificial Neural Networks, to simulators is the ability to extract novel insights into the composites of the surgical performance that differentiate levels of expertise. OBJECTIVE: This study aims to demonstrate the benefits of artificial neural network algorithms in assessing and analyzing virtual surgical performances. This study applies the algorithm on a virtual reality simulated annulus incision task during an anterior cervical discectomy and fusion scenario. DESIGN: An artificial neural network algorithm was developed and integrated. Participants performed the simulated surgical procedure on the Sim-Ortho simulator. Data extracted from the annulus incision task were extracted to generate 157 surgical performance metrics that spanned three categories (motion, safety, and efficiency). SETTING: Musculoskeletal Biomechanics Research Lab; Neurosurgical Simulation and Artificial Intelligence Learning Center, McGill University, Montreal, Canada. PARTICIPANTS: Twenty-three participants were recruited and divided into 3 groups: 11 post-residents, 5 senior and 7 junior residents. RESULTS: An artificial neural network model was trained on nine selected surgical metrics, spanning all three categories and achieved 80% testing accuracy. CONCLUSIONS: This study outlines the benefits of integrating artificial neural networks to virtual reality surgical simulators in understanding composites of expertise performance.
Assuntos
Realidade Virtual , Inteligência Artificial , Competência Clínica , Simulação por Computador , Humanos , Redes Neurais de Computação , Interface Usuário-ComputadorRESUMO
BACKGROUND: Animal brain tumor models can be useful educational tools for the training of neurosurgical residents in risk-free environments. Magnetic resonance imaging (MRI) technologies have not used these models to quantitate tumor, normal gray and white matter, and total tissue removal during complex neurosurgical procedures. This pilot study was carried out as a proof of concept to show the feasibility of using brain tumor models combined with 7-T MRI technology to quantitatively assess tissue removal during subpial tumor resection. METHODS: Seven ex vivo calf brain hemispheres were used to develop the 7-T MRI segmentation methodology. Three brains were used to quantitate brain tissue removal using 7-T MRI segmentation methodology. Alginate artificial brain tumor was created in 4 calf brains to assess the ability of 7-T MRI segmentation methodology to quantitate tumor and gray and white matter along with total tissue volumes removal during a subpial tumor resection procedure. RESULTS: Quantitative studies showed a correlation between removed brain tissue weights and volumes determined from segmented 7-T MRIs. Analysis of baseline and postresection alginate brain tumor segmented 7-T MRIs allowed quantification of tumor and gray and white matter along with total tissue volumes removed and detection of alterations in surrounding gray and white matter. CONCLUSIONS: This pilot study showed that the use of animal tumor models in combination with 7-T MRI technology provides an opportunity to increase the granularity of data obtained from operative procedures and to improve the assessment and training of learners.
Assuntos
Neoplasias Encefálicas , Córtex Cerebral/diagnóstico por imagem , Modelos Animais de Doenças , Imageamento por Ressonância Magnética/métodos , Alginatos , Animais , Bovinos , Córtex Cerebral/cirurgia , Meios de Contraste , Marcadores Fiduciais , Gadolínio , Substância Cinzenta/diagnóstico por imagem , Neoplasia Residual , Imagens de Fantasmas , Projetos Piloto , Estudo de Prova de Conceito , Especificidade da Espécie , Realidade Virtual , Substância Branca/diagnóstico por imagemRESUMO
Simulation-based training is increasingly being used for assessment and training of psychomotor skills involved in medicine. The application of artificial intelligence and machine learning technologies has provided new methodologies to utilize large amounts of data for educational purposes. A significant criticism of the use of artificial intelligence in education has been a lack of transparency in the algorithms' decision-making processes. This study aims to 1) introduce a new framework using explainable artificial intelligence for simulation-based training in surgery, and 2) validate the framework by creating the Virtual Operative Assistant, an automated educational feedback platform. Twenty-eight skilled participants (14 staff neurosurgeons, 4 fellows, 10 PGY 4-6 residents) and 22 novice participants (10 PGY 1-3 residents, 12 medical students) took part in this study. Participants performed a virtual reality subpial brain tumor resection task on the NeuroVR simulator using a simulated ultrasonic aspirator and bipolar. Metrics of performance were developed, and leave-one-out cross validation was employed to train and validate a support vector machine in Matlab. The classifier was combined with a unique educational system to build the Virtual Operative Assistant which provides users with automated feedback on their metric performance with regards to expert proficiency performance benchmarks. The Virtual Operative Assistant successfully classified skilled and novice participants using 4 metrics with an accuracy, specificity and sensitivity of 92, 82 and 100%, respectively. A 2-step feedback system was developed to provide participants with an immediate visual representation of their standing related to expert proficiency performance benchmarks. The educational system outlined establishes a basis for the potential role of integrating artificial intelligence and virtual reality simulation into surgical educational teaching. The potential of linking expertise classification, objective feedback based on proficiency benchmarks, and instructor input creates a novel educational tool by integrating these three components into a formative educational paradigm.
Assuntos
Inteligência Artificial , Neurocirurgia/educação , Treinamento por Simulação/métodos , Realidade Virtual , Algoritmos , Neoplasias Encefálicas/cirurgia , Simulação por Computador , Educação Médica/métodos , Feminino , Humanos , Masculino , Máquina de Vetores de Suporte , Cirurgia Assistida por Computador/educaçãoRESUMO
BACKGROUND: Virtual reality surgical simulators provide a safe environment for trainees to practice specific surgical scenarios and allow for self-guided learning. Artificial intelligence technology, including artificial neural networks, offers the potential to manipulate large datasets from simulators to gain insight into the importance of specific performance metrics during simulated operative tasks. OBJECTIVE: To distinguish performance in a virtual reality-simulated anterior cervical discectomy scenario, uncover novel performance metrics, and gain insight into the relative importance of each metric using artificial neural networks. METHODS: Twenty-one participants performed a simulated anterior cervical discectomy on the novel virtual reality Sim-Ortho simulator. Participants were divided into 3 groups, including 9 post-resident, 5 senior, and 7 junior participants. This study focused on the discectomy portion of the task. Data were recorded and manipulated to calculate metrics of performance for each participant. Neural networks were trained and tested and the relative importance of each metric was calculated. RESULTS: A total of 369 metrics spanning 4 categories (safety, efficiency, motion, and cognition) were generated. An artificial neural network was trained on 16 selected metrics and tested, achieving a training accuracy of 100% and a testing accuracy of 83.3%. Network analysis identified safety metrics, including the number of contacts on spinal dura, as highly important. CONCLUSION: Artificial neural networks classified 3 groups of participants based on expertise allowing insight into the relative importance of specific metrics of performance. This novel methodology aids in the understanding of which components of surgical performance predominantly contribute to expertise.
Assuntos
Realidade Virtual , Inteligência Artificial , Competência Clínica , Discotomia , Humanos , Redes Neurais de ComputaçãoRESUMO
BACKGROUND: Virtual reality spine simulators are emerging as potential educational tools to assess and train surgical procedures in safe environments. Analysis of validity is important in determining the educational utility of these systems. OBJECTIVE: To assess face, content, and construct validity of a C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho virtual reality platform, developed by OSSimTechTM (Montreal, Canada) and the AO Foundation (Davos, Switzerland). METHODS: Spine surgeons, spine fellows, along with neurosurgical and orthopedic residents, performed a simulated C4-C5 anterior cervical discectomy and fusion on the Sim-Ortho system. Participants were separated into 3 categories: post-residents (spine surgeons and spine fellows), senior residents, and junior residents. A Likert scale was used to assess face and content validity. Construct validity was evaluated by investigating differences between the 3 groups on metrics derived from simulator data. The Kruskal-Wallis test was employed to compare groups and a post-hoc Dunn's test with a Bonferroni correction was utilized to investigate differences between groups on significant metrics. RESULTS: A total of 21 individuals were included: 9 post-residents, 5 senior residents, and 7 junior residents. The post-resident group rated face and content validity, median ≥4, for the overall procedure and at least 1 tool in each of the 4 steps. Significant differences (P < .05) were found between the post-resident group and senior and/or junior residents on at least 1 metric for each component of the simulation. CONCLUSION: The C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho platform demonstrated face, content, and construct validity suggesting its utility as a formative educational tool.
Assuntos
Realidade Virtual , Simulação por Computador , Discotomia , HumanosRESUMO
BACKGROUND: The operative environment poses many challenges to studying the relationship between surgical acts and patient outcomes in intracranial oncological neurosurgery. We sought to develop a framework in which neurosurgical performance and extent of resection could be precisely quantified in a controlled setting. METHODS: The stiffness of an alginate hydrogel-based tumor was modified with differing concentrations of the cross-linking agent calcium sulfate until biomechanical properties similar to those of human primary brain tumors measured at resection were achieved. The artificial tumor was subsequently incorporated into an ex-vivo animal brain as a final model. Magnetic resonance imaging enhancement and ultraviolet fluorescence was achieved by incorporating gadolinium and fluorescein solution, respectively. Video recordings from the operative microscope, ceiling cameras, and instrument-mounted fiducial markers within a surgical suite environment captured operative performance. RESULTS: A total of 24 rheometer measurements were conducted on alginate hydrogels containing 10-, 11-, and 12-mM concentrations of calcium sulfate. Sixty-eight stiffness measurements were conducted on eight patient tumor samples. No differences were found between the alginate and brain tumor stiffness values [Kruskal-Wallis χ2(4) = 9.187; P = 0.057]. Tumor was identified using ultraviolet fluorescence and ultrasonography. The volume and location of the resected white and gray matter and residual tumor could be quantified in 0.003-mm3 increments using a 7T magnetic resonance imaging coil. Ultrasonic aspirator and bipolar electrocautery movement data were successfully transformed into performance metrics. CONCLUSION: The developed framework can offer clinicians, learners, and researchers the ability to perform operative rehearsal, teaching, and studies involving brain tumor surgery in a controlled laboratory environment and represents a crucial step in the understanding and training of expertise in neurosurgery.
Assuntos
Neoplasias Encefálicas/cirurgia , Procedimentos Neurocirúrgicos/métodos , Projetos de Pesquisa , Alginatos , Animais , Fenômenos Biomecânicos , Neoplasias Encefálicas/diagnóstico por imagem , Sulfato de Cálcio , Bovinos , Simulação por Computador , Reagentes de Ligações Cruzadas , Fluorescência , Humanos , Hidrogéis , Imageamento por Ressonância Magnética , Modelos Anatômicos , Resultado do Tratamento , Ultrassonografia , Gravação em VídeoRESUMO
Importance: Despite advances in the assessment of technical skills in surgery, a clear understanding of the composites of technical expertise is lacking. Surgical simulation allows for the quantitation of psychomotor skills, generating data sets that can be analyzed using machine learning algorithms. Objective: To identify surgical and operative factors selected by a machine learning algorithm to accurately classify participants by level of expertise in a virtual reality surgical procedure. Design, Setting, and Participants: Fifty participants from a single university were recruited between March 1, 2015, and May 31, 2016, to participate in a case series study at McGill University Neurosurgical Simulation and Artificial Intelligence Learning Centre. Data were collected at a single time point and no follow-up data were collected. Individuals were classified a priori as expert (neurosurgery staff), seniors (neurosurgical fellows and senior residents), juniors (neurosurgical junior residents), and medical students, all of whom participated in 250 simulated tumor resections. Exposures: All individuals participated in a virtual reality neurosurgical tumor resection scenario. Each scenario was repeated 5 times. Main Outcomes and Measures: Through an iterative process, performance metrics associated with instrument movement and force, resection of tissues, and bleeding generated from the raw simulator data output were selected by K-nearest neighbor, naive Bayes, discriminant analysis, and support vector machine algorithms to most accurately determine group membership. Results: A total of 50 individuals (9 women and 41 men; mean [SD] age, 33.6 [9.5] years; 14 neurosurgeons, 4 fellows, 10 senior residents, 10 junior residents, and 12 medical students) participated. Neurosurgeons were in practice between 1 and 25 years, with 9 (64%) involving a predominantly cranial practice. The K-nearest neighbor algorithm had an accuracy of 90% (45 of 50), the naive Bayes algorithm had an accuracy of 84% (42 of 50), the discriminant analysis algorithm had an accuracy of 78% (39 of 50), and the support vector machine algorithm had an accuracy of 76% (38 of 50). The K-nearest neighbor algorithm used 6 performance metrics to classify participants, the naive Bayes algorithm used 9 performance metrics, the discriminant analysis algorithm used 8 performance metrics, and the support vector machine algorithm used 8 performance metrics. Two neurosurgeons, 1 fellow or senior resident, 1 junior resident, and 1 medical student were misclassified. Conclusions and Relevance: In a virtual reality neurosurgical tumor resection study, a machine learning algorithm successfully classified participants into 4 levels of expertise with 90% accuracy. These findings suggest that algorithms may be capable of classifying surgical expertise with greater granularity and precision than has been previously demonstrated in surgery.