RESUMO
PURPOSE: The modern operating room is becoming increasingly complex, requiring innovative intra-operative support systems. While the focus of surgical data science has largely been on video analysis, integrating surgical computer vision with natural language capabilities is emerging as a necessity. Our work aims to advance visual question answering (VQA) in the surgical context with scene graph knowledge, addressing two main challenges in the current surgical VQA systems: removing question-condition bias in the surgical VQA dataset and incorporating scene-aware reasoning in the surgical VQA model design. METHODS: First, we propose a surgical scene graph-based dataset, SSG-VQA, generated by employing segmentation and detection models on publicly available datasets. We build surgical scene graphs using spatial and action information of instruments and anatomies. These graphs are fed into a question engine, generating diverse QA pairs. We then propose SSG-VQA-Net, a novel surgical VQA model incorporating a lightweight Scene-embedded Interaction Module, which integrates geometric scene knowledge in the VQA model design by employing cross-attention between the textual and the scene features. RESULTS: Our comprehensive analysis shows that our SSG-VQA dataset provides a more complex, diverse, geometrically grounded, unbiased and surgical action-oriented dataset compared to existing surgical VQA datasets and SSG-VQA-Net outperforms existing methods across different question types and complexities. We highlight that the primary limitation in the current surgical VQA systems is the lack of scene knowledge to answer complex queries. CONCLUSION: We present a novel surgical VQA dataset and model and show that results can be significantly improved by incorporating geometric scene features in the VQA model design. We point out that the bottleneck of the current surgical visual question-answer model lies in learning the encoded representation rather than decoding the sequence. Our SSG-VQA dataset provides a diagnostic benchmark to test the scene understanding and reasoning capabilities of the model. The source code and the dataset will be made publicly available at: https://github.com/CAMMA-public/SSG-VQA .
Assuntos
Salas Cirúrgicas , Humanos , Cirurgia Assistida por Computador/métodos , Processamento de Linguagem Natural , Gravação em VídeoRESUMO
The field of surgical computer vision has undergone considerable breakthroughs in recent years with the rising popularity of deep neural network-based methods. However, standard fully-supervised approaches for training such models require vast amounts of annotated data, imposing a prohibitively high cost; especially in the clinical domain. Self-Supervised Learning (SSL) methods, which have begun to gain traction in the general computer vision community, represent a potential solution to these annotation costs, allowing to learn useful representations from only unlabeled data. Still, the effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. In this work, we address this critical need by investigating four state-of-the-art SSL methods (MoCo v2, SimCLR, DINO, SwAV) in the context of surgical computer vision. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection. We examine their parameterization, then their behavior with respect to training data quantities in semi-supervised settings. Correct transfer of these methods to surgery, as described and conducted in this work, leads to substantial performance gains over generic uses of SSL - up to 7.4% on phase recognition and 20% on tool presence detection - as well as state-of-the-art semi-supervised phase recognition approaches by up to 14%. Further results obtained on a highly diverse selection of surgical datasets exhibit strong generalization properties. The code is available at https://github.com/CAMMA-public/SelfSupSurg.
Assuntos
Computadores , Redes Neurais de Computação , Humanos , Aprendizado de Máquina SupervisionadoRESUMO
The fine-grained localization of clinicians in the operating room (OR) is a key component to design the new generation of OR support systems. Computer vision models for person pixel-based segmentation and body-keypoints detection are needed to better understand the clinical activities and the spatial layout of the OR. This is challenging, not only because OR images are very different from traditional vision datasets, but also because data and annotations are hard to collect and generate in the OR due to privacy concerns. To address these concerns, we first study how joint person pose estimation and instance segmentation can be performed on low resolutions images with downsampling factors from 1x to 12x. Second, to address the domain shift and the lack of annotations, we propose a novel unsupervised domain adaptation method, called AdaptOR, to adapt a model from an in-the-wild labeled source domain to a statistically different unlabeled target domain. We propose to exploit explicit geometric constraints on the different augmentations of the unlabeled target domain image to generate accurate pseudo labels and use these pseudo labels to train the model on high- and low-resolution OR images in a self-training framework. Furthermore, we propose disentangled feature normalization to handle the statistically different source and target domain data. Extensive experimental results with detailed ablation studies on the two OR datasets MVOR+ and TUM-OR-test show the effectiveness of our approach against strongly constructed baselines, especially on the low-resolution privacy-preserving OR images. Finally, we show the generality of our method as a semi-supervised learning (SSL) method on the large-scale COCO dataset, where we achieve comparable results with as few as 1% of labeled supervision against a model trained with 100% labeled supervision. Code is available at https://github.com/CAMMA-public/HPE-AdaptOR.
Assuntos
Redes Neurais de Computação , Salas Cirúrgicas , Humanos , Processamento de Imagem Assistida por Computador/métodos , Software , Aprendizado de Máquina SupervisionadoRESUMO
PURPOSE: Face detection is a needed component for the automatic analysis and assistance of human activities during surgical procedures. Efficient face detection algorithms can indeed help to detect and identify the persons present in the room and also be used to automatically anonymize the data. However, current algorithms trained on natural images do not generalize well to the operating room (OR) images. In this work, we provide a comparison of state-of-the-art face detectors on OR data and also present an approach to train a face detector for the OR by exploiting non-annotated OR images. METHODS: We propose a comparison of six state-of-the-art face detectors on clinical data using multi-view OR faces, a dataset of OR images capturing real surgical activities. We then propose to use self-supervision, a domain adaptation method, for the task of face detection in the OR. The approach makes use of non-annotated images to fine-tune a state-of-the-art detector for the OR without using any human supervision. RESULTS: The results show that the best model, namely the tiny face detector, yields an average precision of 0.556 at intersection over union of 0.5. Our self-supervised model using non-annotated clinical data outperforms this result by 9.2%. CONCLUSION: We present the first comparison of state-of-the-art face detectors on OR images and show that results can be significantly improved by using self-supervision on non-annotated data.
Assuntos
Algoritmos , Face , Salas Cirúrgicas , Reconhecimento Facial , Humanos , SoftwareRESUMO
BACKGROUND: Box trainers are ideal simulators, given they are inexpensive, accessible, and use appropriate fidelity. OBJECTIVE: The development and validation of an open-source, partial task simulator that teaches the fundamental skills necessary for endonasal skull-base neuro-endoscopic surgery. METHODS: We defined the Neuro-Endo-Trainer (NET) SkullBase-Task-GraspPickPlace with an activity area by analyzing the computed tomography scans of 15 adult patients with sellar suprasellar parasellar tumors. Four groups of participants (Group E, n = 4: expert neuroendoscopists; Group N, n =19: novice neurosurgeons; Group R, n = 11: neurosurgery residents with multiple iterations; and Group T, n = 27: neurosurgery residents with single iteration) performed grasp, pick, and place tasks using NET and were graded on task completion time and skills assessment scale score. RESULTS: Group E had lower task completion times and greater skills assessment scale scores than both Group N and R (P ≤ 0.03, 0.001). The performance of Groups N and R was found to be equivalent; in self-assessing neuro-endoscopic skill, the participants in these groups were found to have equally low pretraining scores (4/10) with significant improvement shown after NET simulation (6, 7 respectively). Angled scopes resulted in decreased scores with tilted plates compared with straight plates (30° P ≤ 0.04, 45° P ≤ 0.001). With tilted plates, decreased scores were observed when we compared the 0° with 45° endoscope (right, P ≤ 0.008; left, P ≤ 0.002). CONCLUSIONS: The NET, a face and construct valid open-source partial task neuroendoscopic trainer, was designed. Presimulation novice neurosurgeons and neurosurgical residents were described as having insufficient skills and preparation to practice neuro-endoscopy. Plate tilt and endoscope angle were shown to be important factors in participant performance. The NET was found to be a useful partial-task trainer for skill building in neuro-endoscopy.
Assuntos
Endoscopia/educação , Cavidade Nasal/cirurgia , Neuroendoscopia/educação , Neurocirurgia/educação , Adolescente , Adulto , Competência Clínica , Simulação por Computador , Avaliação Educacional , Feminino , Humanos , Índia , Internato e Residência , Masculino , Manequins , Pessoa de Meia-Idade , Neoplasias Hipofisárias/patologia , Neoplasias Hipofisárias/cirurgia , Reprodutibilidade dos Testes , Autoavaliação (Psicologia) , Base do Crânio/cirurgia , Cirurgiões , Tomografia Computadorizada por Raios X , Adulto JovemRESUMO
OBJECT: The surgical corridor to the upper third of the clivus and ventral brainstem is hindered by critical neurovascular structures, such as the cavernous sinus, petrous apex, and tentorium. The traditional Kawase approach provides a 10 × 5-mm fenestration at the petrous apex of the temporal bone between the 5th cranial nerve and internal auditory canal. Due to interindividual variability, sometimes this area proves to be insufficient as a corridor to the posterior cranial fossa. The authors describe a modification to the technique of the extradural anterior petrosectomy consisting of additional transcavernous exploration and medial mobilization of the cisternal component of the trigeminal nerve. This approach is termed the modified Dolenc-Kawase (MDK) approach. METHODS: The authors describe a volumetric analysis of temporal bones with 3D laser scanning of dry and drilled bones for respective triangles and rhomboid areas, and they compare the difference of exposure with traditional versus modified approaches on cadaver dissection. Twelve dry temporal bones were laser scanned, and mesh-based volumetric analysis was done followed by drilling of the Kawase triangle and MDK rhomboid. Five cadaveric heads were drilled on alternate sides with both approaches for evaluation of the area exposed, surgical freedom, and angle of approach. RESULTS: The MDK approach provides an approximately 1.5 times larger area and 2.0 times greater volume of bone at the anterior petrous apex compared with the Kawase's approach. Cadaver dissection objectified the technical feasibility of the MDK approach, providing nearly 1.5-2 times larger fenestration with improved view and angulation to the posterior cranial fossa. Practical application in 6 patients with different lesions proves clinical applicability of the MDK approach. CONCLUSIONS: The larger fenestration at the petrous apex achieved with the MDK approach provides greater surgical freedom at the Dorello canal, gasserian ganglion, and prepontine area and better anteroposterior angulation than the traditional Kawase approach. Additional anterior clinoidectomy and transcavernous exposure helps in dealing with basilar artery aneurysms.
Assuntos
Fossa Craniana Média/cirurgia , Procedimentos Neurocirúrgicos/métodos , Osso Petroso/cirurgia , Neoplasias da Base do Crânio/cirurgia , Cadáver , Fossa Craniana Posterior/cirurgia , Humanos , Imageamento Tridimensional , Osso Temporal/cirurgiaRESUMO
BACKGROUND: Drilling of the anterior clinoid process (ACP) is an integral component of surgical approaches for central and paracentral skull base lesions. The technique to drill ACP has evolved from pure intradural to extradural and combined techniques. OBJECTIVE: To describe the computerized morphometric evaluation of exposure of optic nerve and internal carotid artery with proposed tailored intradural (IDAC) and complete extradural (EDAC) anterior clinoidectomy. METHODS: We describe a morphometric subdivision of ACP into 4 quadrangles and 1 triangle on the basis of fixed bony landmarks. Computerized volumetric analysis with 3-dimensional laser scanning of dry-drilled bones for respective tailored IDAC and EDAC was performed. Both approaches were compared for the area and length of the optic nerve and internal carotid artery. Five cadaver heads were dissected on alternate sides with intradural and extradural techniques to evaluate exposure, surgical freedom, and angulation of approach. RESULTS: Complete anterior clinoidectomy provides a 2.5-times larger area and 2.7-times larger volume of ACP. Complete clinoidectomy deroofed the optic nerve to an equal extent as by proposed the partial tailored clinoidectomy approach. Tailored IDAC exposes only the distal dural ring, whereas complete EDAC exposes both the proximal and distal dural rings with complete exposure of the carotid cave. CONCLUSION: Quantitative comparative evaluation provides details of exposure and surgical ease with both techniques. We promote hybrid/EDAC technique for vascular pathologies because of better anatomic orientation. Extradural clinoidectomy is the preferred technique for midline cranial neoplasia. An awareness of different variations of clinoidectomy can prevent dependency on any particular approach and facilitate flexibility.
Assuntos
Base do Crânio/anatomia & histologia , Base do Crânio/cirurgia , Osso Esfenoide/anatomia & histologia , Osso Esfenoide/cirurgia , Cadáver , Artéria Carótida Interna/cirurgia , Humanos , Imageamento TridimensionalRESUMO
AIM: The utilization of technology for purpose of imparting knowledge, especially in high-end branches like neurosurgery, has gained prominence in the contemporary academic scenario. The technological advancements have brought about outstanding transformation to education and patient care. The connectivity through smartphone applications (apps) has transcended the spatial and temporal limitations, thereby enabling easy access to virtually infinite storehouse of knowledge. Although there are numerous neurosurgery related apps, yet there is still a dearth of quality apps that may serve the purpose. MATERIAL AND METHODS: Relevant apps were searched and evaluated on PlayStore and Apple App store, based on their content, user interface, performance, and utility in routine practice and compared with their cost, size and popularity. They were categorized into apps related to textbooks, scoring systems, patient education, operative procedures, blogs, journals, conferences and promotional. RESULTS: 159 relevant apps were hosted on App stores; 54.7% apps were free of cost. "Neuromind" was the most downloaded app because of its simplicity, free access and applicability. Students and practitioners prefer various apps linked to scoring systems, textbooks and operative illustrations. Apps have helped patients in better understanding of their diseases and management options. CONCLUSION: Development of web-based technologies has divided medical professionals into traditional and modernized learners. Mobile apps permit knowledge to be structured visually to facilitate its easy diffusion in the peer community. A technologically demanding branch like Neurosurgery inevitably needs innovative, cost effective apps with trust worthy content. Relevant apps have a high potential to be used as an excellent resource for effectual neurosurgical education beyond the limitations of time and place.
Assuntos
Aplicativos Móveis , Neurocirurgia , HumanosRESUMO
BACKGROUND: Since the end of last century, technology has taken a front seat in dispersion of medical education. Advancements of technology in neurosurgery and traditional training methods are now being challenged by legal and ethical concerns of patient safety, resident work-hour restriction and cost of operating-room time. To supplement the existing neurosurgery education pattern, various e-learning platforms are introduced as structured, interactive learning system. MATERIALS AND METHODS: This study focuses on the concept, formulation, development and impact of web based learning platforms dedicated to neurosurgery discipline to disseminate education, supplement surgical knowledge and improve skills of neurosurgeons. 'Neurosurgery Education and Training School (NETS), e-learning platform' has integration of web-based technologies like 'Content Management System' for organizing the education material and 'Learning Management System' for updating neurosurgeons. NETS discussion forum networks neurosurgeons, neuroscientists and neuro-technologists across the globe facilitating collaborative translational research. RESULTS: Multi-authored neurosurgical e-learning material supplements the deficiencies of regular time-bound education. Interactive open-source, global, free-access e-learning platform of NETS has around 1) 425 visitors/month from 73 countries; ratio of new visitors to returning visitors 42.3; 57.7 (2); 64,380 views from 190 subscribers for surgical videos, 3-D animation, graphics based training modules (3); average 402 views per post. CONCLUSION: The e-Learning platforms provide updated educational content that make them "quick, surf, find and extract" resources. e-Learning tools like web-based education, social interactive platform and question-answer forum will save unnecessary expenditure of time and travel of neurosurgeons seeking knowledge. The need for free access platforms is more pronounced for the neurosurgeons and patients in developing nations.