Fully immersive virtual reality for skull-base surgery: surgical training and beyond.

PURPOSE: A virtual reality (VR) system, where surgeons can practice procedures on virtual anatomies, is a scalable and cost-effective alternative to cadaveric training. The fully digitized virtual surgeries can also be used to assess the surgeon's skills using measurements that are otherwise hard to collect in reality. Thus, we present the Fully Immersive Virtual Reality System (FIVRS) for skull-base surgery, which combines surgical simulation software with a high-fidelity hardware setup. METHODS: FIVRS allows surgeons to follow normal clinical workflows inside the VR environment. FIVRS uses advanced rendering designs and drilling algorithms for realistic bone ablation. A head-mounted display with ergonomics similar to that of surgical microscopes is used to improve immersiveness. Extensive multi-modal data are recorded for post-analysis, including eye gaze, motion, force, and video of the surgery. A user-friendly interface is also designed to ease the learning curve of using FIVRS. RESULTS: We present results from a user study involving surgeons with various levels of expertise. The preliminary data recorded by FIVRS differentiate between participants with different levels of expertise, promising future research on automatic skill assessment. Furthermore, informal feedback from the study participants about the system's intuitiveness and immersiveness was positive. CONCLUSION: We present FIVRS, a fully immersive VR system for skull-base surgery. FIVRS features a realistic software simulation coupled with modern hardware for improved realism. The system is completely open source and provides feature-rich data in an industry-standard format.

TAToo: vision-based joint tracking of anatomy and tool for skull-base surgery.

PURPOSE: Tracking the 3D motion of the surgical tool and the patient anatomy is a fundamental requirement for computer-assisted skull-base surgery. The estimated motion can be used both for intra-operative guidance and for downstream skill analysis. Recovering such motion solely from surgical videos is desirable, as it is compliant with current clinical workflows and instrumentation. METHODS: We present Tracker of Anatomy and Tool (TAToo). TAToo jointly tracks the rigid 3D motion of the patient skull and surgical drill from stereo microscopic videos. TAToo estimates motion via an iterative optimization process in an end-to-end differentiable form. For robust tracking performance, TAToo adopts a probabilistic formulation and enforces geometric constraints on the object level. RESULTS: We validate TAToo on both simulation data, where ground truth motion is available, as well as on anthropomorphic phantom data, where optical tracking provides a strong baseline. We report sub-millimeter and millimeter inter-frame tracking accuracy for skull and drill, respectively, with rotation errors below [Formula: see text]. We further illustrate how TAToo may be used in a surgical navigation setting. CONCLUSIONS: We present TAToo, which simultaneously tracks the surgical tool and the patient anatomy in skull-base surgery. TAToo directly predicts the motion from surgical videos, without the need of any markers. Our results show that the performance of TAToo compares favorably to competing approaches. Future work will include fine-tuning of our depth network to reach a 1 mm clinical accuracy goal desired for surgical applications in the skull base.

Twin-S: a digital twin for skull base surgery.

PURPOSE: Digital twins are virtual replicas of real-world objects and processes, and they have potential applications in the field of surgical procedures, such as enhancing situational awareness. We introduce Twin-S, a digital twin framework designed specifically for skull base surgeries. METHODS: Twin-S is a novel framework that combines high-precision optical tracking and real-time simulation, making it possible to integrate it into image-guided interventions. To guarantee accurate representation, Twin-S employs calibration routines to ensure that the virtual model precisely reflects all real-world processes. Twin-S models and tracks key elements of skull base surgery, including surgical tools, patient anatomy, and surgical cameras. Importantly, Twin-S mirrors real-world drilling and updates the virtual model at frame rate of 28. RESULTS: Our evaluation of Twin-S demonstrates its accuracy, with an average error of 1.39 mm during the drilling process. Our study also highlights the benefits of Twin-S, such as its ability to provide augmented surgical views derived from the continuously updated virtual model, thus offering additional situational awareness to the surgeon. CONCLUSION: We present Twin-S, a digital twin environment for skull base surgery. Twin-S captures the real-world surgical progresses and updates the virtual model in real time through the use of modern tracking technologies. Future research that integrates vision-based techniques could further increase the accuracy of Twin-S.

Rethinking causality-driven robot tool segmentation with temporal constraints.

PURPOSE: Vision-based robot tool segmentation plays a fundamental role in surgical robots perception and downstream tasks. CaRTS, based on a complementary causal model, has shown promising performance in unseen counterfactual surgical environments in the presence of smoke, blood, etc. However, CaRTS requires over 30 iterations of optimization to converge for a single image due to limited observability. METHOD: To address the above limitations, we take temporal relation into consideration and propose a temporal causal model for robot tool segmentation on video sequences. We design an architecture named Temporally Constrained CaRTS (TC-CaRTS). TC-CaRTS has three novel modules to complement CaRTS-temporal optimization pipeline, kinematics correction network, and spatial-temporal regularization. RESULTS: Experiment results show that TC-CaRTS requires fewer iterations to achieve the same or better performance as CaRTS on different domains. All three modules are proven to be effective. CONCLUSION: We propose TC-CaRTS, which takes advantage of temporal constraints as additional observability. We show that TC-CaRTS outperforms prior work in the robot tool segmentation task with improved convergence speed on test datasets from different domains.

A Self-Configuring Deep Learning Network for Segmentation of Temporal Bone Anatomy in Cone-Beam CT Imaging.

OBJECTIVE: Preoperative planning for otologic or neurotologic procedures often requires manual segmentation of relevant structures, which can be tedious and time-consuming. Automated methods for segmenting multiple geometrically complex structures can not only streamline preoperative planning but also augment minimally invasive and/or robot-assisted procedures in this space. This study evaluates a state-of-the-art deep learning pipeline for semantic segmentation of temporal bone anatomy. STUDY DESIGN: A descriptive study of a segmentation network. SETTING: Academic institution. METHODS: A total of 15 high-resolution cone-beam temporal bone computed tomography (CT) data sets were included in this study. All images were co-registered, with relevant anatomical structures (eg, ossicles, inner ear, facial nerve, chorda tympani, bony labyrinth) manually segmented. Predicted segmentations from no new U-Net (nnU-Net), an open-source 3-dimensional semantic segmentation neural network, were compared against ground-truth segmentations using modified Hausdorff distances (mHD) and Dice scores. RESULTS: Fivefold cross-validation with nnU-Net between predicted and ground-truth labels were as follows: malleus (mHD: 0.044 ± 0.024 mm, dice: 0.914 ± 0.035), incus (mHD: 0.051 ± 0.027 mm, dice: 0.916 ± 0.034), stapes (mHD: 0.147 ± 0.113 mm, dice: 0.560 ± 0.106), bony labyrinth (mHD: 0.038 ± 0.031 mm, dice: 0.952 ± 0.017), and facial nerve (mHD: 0.139 ± 0.072 mm, dice: 0.862 ± 0.039). Comparison against atlas-based segmentation propagation showed significantly higher Dice scores for all structures (p < .05). CONCLUSION: Using an open-source deep learning pipeline, we demonstrate consistently submillimeter accuracy for semantic CT segmentation of temporal bone anatomy compared to hand-segmented labels. This pipeline has the potential to greatly improve preoperative planning workflows for a variety of otologic and neurotologic procedures and augment existing image guidance and robot-assisted systems for the temporal bone.

Transcriptional factor 3 binds to sirtuin 1 to activate the Wnt/ß-catenin signaling in cervical cancer.

Transcriptional factor 3 (TCF3, also termed E2A), first reported to exert crucial functions during lymphocyte development, has been revealed to participate in the pathogenesis of human cancers. The aim of this work was to investigate the function of TCF3 in cervical cancer (CC) and the molecular interactions. The bioinformatics prediction suggested that TCF3 was highly expressed in CC and linked to poor prognosis. Increased TCF3 expression was identified in CC cell lines, and its downregulation reduced proliferation and migration of CC cells in vitro as well as growth of xenograft tumors in vivo. Gene Ontology and Kyoto Encyclopedia of Genes and Genomes enrichment analyses showed that the TCF-3-related genes and genes showed differential expression between CC and normal tissues were mainly enriched in the Wnt/ß-catenin pathway. TCF3 bound to sirtuin 1 (SIRT1) promoter for transcriptional activation, and SIRT1 promoted deacetylation and nuclear translocation of ß-catenin in CC. SIRT1 overexpression blocked the role of TCF3 silencing and restored cell proliferation in vitro and tumor growth in vivo. Treatment with XAV-939, a ß-catenin inhibitor, significantly suppressed the cell proliferation and tumor growth induced by SIRT1 overexpression. In conclusion, this study demonstrates that TCF3 augments progression of CC by activating SIRT1-mediated ß-catenin signaling.

Automated Extraction of Anatomical Measurements From Temporal Bone CT Imaging.

OBJECTIVE: Proposed methods of minimally invasive and robot-assisted procedures within the temporal bone require measurements of surgically relevant distances and angles, which often require time-consuming manual segmentation of preoperative imaging. This study aims to describe an automatic segmentation and measurement extraction pipeline of temporal bone cone-beam computed tomography (CT) scans. STUDY DESIGN: Descriptive study of temporal bone measurements. SETTING: Academic institution. METHODS: A propagation template composed of 16 temporal bone CT scans was formed with relevant anatomical structures and landmarks manually segmented. Next, 52 temporal bone CT scans were autonomously segmented using deformable registration techniques from the Advanced Normalization Tools Python package. Anatomical measurements were extracted via in-house Python algorithms. Extracted measurements were compared to ground truth values from manual segmentations. RESULTS: Paired t test analyses showed no statistical difference between measurements using this pipeline and ground truth measurements from manually segmented images. Mean (SD) malleus manubrium length was 4.39 (0.34) mm. Mean (SD) incus short and long processes were 2.91 (0.18) mm and 3.53 (0.38) mm, respectively. The mean (SD) maximal diameter of the incus long process was 0.74 (0.17) mm. The first and second facial nerve genus had mean (SD) angles of 68.6 (6.7) degrees and 111.1 (5.3) degrees, respectively. The facial recess had a mean (SD) span of 3.21 (0.46) mm. Mean (SD) minimum distance between the external auditory canal and tegmen was 3.79 (1.05) mm. CONCLUSIONS: This is the first study to automatically extract relevant temporal bone anatomical measurements from CT scans using segmentation propagation. Measurements from these models can streamline preoperative planning, improve future segmentation techniques, and help develop future image-guided or robot-assisted systems for temporal bone procedures.

Automated Registration-Based Temporal Bone Computed Tomography Segmentation for Applications in Neurotologic Surgery.

OBJECTIVE: This study investigates the accuracy of an automated method to rapidly segment relevant temporal bone anatomy from cone beam computed tomography (CT) images. Implementation of this segmentation pipeline has potential to improve surgical safety and decrease operative time by augmenting preoperative planning and interfacing with image-guided robotic surgical systems. STUDY DESIGN: Descriptive study of predicted segmentations. SETTING: Academic institution. METHODS: We have developed a computational pipeline based on the symmetric normalization registration method that predicts segmentations of anatomic structures in temporal bone CT scans using a labeled atlas. To evaluate accuracy, we created a data set by manually labeling relevant anatomic structures (eg, ossicles, labyrinth, facial nerve, external auditory canal, dura) for 16 deidentified high-resolution cone beam temporal bone CT images. Automated segmentations from this pipeline were compared against ground-truth manual segmentations by using modified Hausdorff distances and Dice scores. Runtimes were documented to determine the computational requirements of this method. RESULTS: Modified Hausdorff distances and Dice scores between predicted and ground-truth labels were as follows: malleus (0.100 ± 0.054 mm; Dice, 0.827 ± 0.068), incus (0.100 ± 0.033 mm; Dice, 0.837 ± 0.068), stapes (0.157 ± 0.048 mm; Dice, 0.358 ± 0.100), labyrinth (0.169 ± 0.100 mm; Dice, 0.838 ± 0.060), and facial nerve (0.522 ± 0.278 mm; Dice, 0.567 ± 0.130). A quad-core 16GB RAM workstation completed this segmentation pipeline in 10 minutes. CONCLUSIONS: We demonstrated submillimeter accuracy for automated segmentation of temporal bone anatomy when compared against hand-segmented ground truth using our template registration pipeline. This method is not dependent on the training data volume that plagues many complex deep learning models. Favorable runtime and low computational requirements underscore this method's translational potential.

Predictors of futile recanalization in patients undergoing endovascular treatment in the DIRECT-MT trial.

BACKGROUND: Futile recanalization-when patients have a successful recanalization but fail to achieve a satisfactory functional outcome- is a common phenomenon of endovascular treatment of acute ischemic stroke (AIS). The present study aimed to identify the predictors of futile recanalization in AIS patients who received endovascular treatment. METHODS: This is a post-hoc analysis of the DIRECT-MT trial. Demographics, clinical characteristics, acute stroke workflow interval times, biochemical parameters, and imaging characteristics were compared between futile and meaningful recanalization groups. Multivariate analysis was performed to identify the predictors of futile recanalization. RESULTS: Futile recanalization was observed in 277 patients. In multivariable logistic regression analysis, older age (p<0.001), higher baseline systolic blood pressure (SBP) (p=0.032), incomplete reperfusion defined by extended Thrombolysis In Cerebral Infarction (eTICI) grades (p=0.020), and larger final infarct volume (FIV) (p<0.001) were independent predictors of futile recanalization. CONCLUSIONS: Old age, high baseline SBP, incomplete reperfusion defined by eTICI, and large FIV were independent predictors of futile recanalization after endovascular therapy for AIS.

Comparing the efficacy and safety of the Skyflow device with those of the Solitaire FR stent in patients with acute ischemic stroke: a prospective, multicenter, randomized, non-inferiority clinical trial.

BACKGROUND: Mechanical thrombectomy is the standard treatment for acute ischemic stroke (AIS) with large vessel occlusion (LVO) in the anterior circulation. This trial aimed to indicate whether Skyflow, a new thrombectomy device, could achieve the same safety and efficacy as Solitaire FR in the treatment of AIS. METHODS: This study was a prospective, multicenter, randomized, single blind, parallel, positive controlled, non-inferiority clinical trial. Patients with intracranial anterior circulation LVO within 8 hours from onset were included to receive thrombectomy treatment with either the Skyflow or Solitaire FR stent retriever. The primary endpoint was the rate of successful reperfusion (modified Treatment In Cerebral Infarction (mTICI) ≥2b) after the operation. The safety endpoints were the rate of symptomatic intracranial hemorrhage (sICH) and subarachnoid hemorrhage (SAH) at 24 hours after operation. RESULTS: A total of 95 and 97 patients were involved in the Skyflow group and Solitaire FR group, respectively. A successful reperfusion (mTICI ≥2b) was finally achieved in 84 (88.4%) patients in the Skyflow group and 80 (82.5%) patients in the Solitaire FR group. Skyflow was non-inferior to Solitaire FR in regard to the primary outcome, with the criterion of a non-inferiority margin of 12.5% (p=0.0002) after being adjusted for the combined center effect and the National Institutes of Health Stroke Scale (NIHSS) score. The rate of periprocedural sICH and SAH did not differ significantly between the two groups. CONCLUSION: Endovascular thrombectomy with the Skyflow stent retriever was non-inferior to Solitaire FR with regard to successful reperfusion in AIS due to LVO (with a pre-specified non-inferiority margin of 12.5%).

Volumetric Accuracy Analysis of Virtual Safety Barriers for Cooperative-Control Robotic Mastoidectomy.

HYPOTHESIS: Virtual fixtures can be enforced in cooperative-control robotic mastoidectomies with submillimeter accuracy. BACKGROUND: Otologic procedures are well-suited for robotic assistance due to consistent osseous landmarks. We have previously demonstrated the feasibility of cooperative-control robots (CCRs) for mastoidectomy. CCRs manipulate instruments simultaneously with the surgeon, allowing the surgeon to control instruments with robotic augmentation of motion. CCRs can also enforce virtual fixtures, which are safety barriers that prevent motion into undesired locations. Previous studies have validated the ability of CCRs to allow a novice surgeon to safely complete a cortical mastoidectomy. This study provides objective accuracy data for CCR-imposed safety barriers in cortical mastoidectomies. METHODS: Temporal bone phantoms were registered to a CCR using preoperative computed tomography (CT) imaging. Virtual fixtures were created using 3D Slicer, with 2D planes placed along the external auditory canal, tegmen, and sigmoid, converging on the antrum. Five mastoidectomies were performed by a novice surgeon, moving the drill to the limit of the barriers. Postoperative CT scans were obtained, and Dice coefficients and Hausdorff distances were calculated. RESULTS: The average modified Hausdorff distance between drilled bone and the preplanned volume was 0.351 ±â€Š0.093 mm. Compared with the preplanned volume of 0.947 cm3, the mean volume of bone removed was 1.045 cm3 (difference of 0.0982 cm3 or 10.36%), with an average Dice coefficient of 0.741 (range, 0.665-0.802). CONCLUSIONS: CCR virtual fixtures can be enforced with a high degree of accuracy. Future studies will focus on improving accuracy and developing 3D fixtures around relevant surgical anatomy.

Safety and efficacy of the SeparGate™ balloon-guiding catheter in neurointerventional surgery: Study protocol of a prospective multicenter single-arm clinical trial.

BACKGROUND: The balloon-guiding catheter (BGC) reportedly reduces the number of retrievals and occurrence of distal emboli, achieving superior revascularization results and improved clinical outcomes in acute stroke. This study will aim to examine the efficacy and safety of the new SeparGate™ BGC. DESIGN: This prospective multicenter single-arm clinical trial will aim to include 128 patients who fulfill its inclusion and exclusion criteria. All patients will receive endovascular interventional therapy with BGC assistance. The primary endpoint will be the immediate surgical success rate, while the secondary endpoint will be product performance. The safety evaluation will include serious adverse events such as puncture site hematoma and bleeding, cerebral vasospasm, vessel dissection, vessel perforation, air embolism, thrombus (acute or subacute), vessel occlusion, distal embolization, infection, adverse reaction to antiplatelet and anticoagulant drugs, intracranial hemorrhage, stroke, death, and device defect. DISCUSSION: The prospective multicenter trial will provide safety and efficacy information for the SeparGate™ BGC. Its findings will provide a clinical reference for endovascular adjuvant therapy of cerebrovascular disease. TRIAL REGISTRATION: ChiCTR1800014459.

A Novel Semi-Autonomous Control Framework for Retina Confocal Endomicroscopy Scanning.

In this paper, a novel semi-autonomous control framework is presented for enabling probe-based confocal laser endomicroscopy (pCLE) scan of the retinal tissue. With pCLE, retinal layers such as nerve fiber layer (NFL) and retinal ganglion cell (RGC) can be scanned and characterized in real-time for an improved diagnosis and surgical outcome prediction. However, the limited field of view of the pCLE system and the micron-scale optimal focus distance of the probe, which are in the order of physiological hand tremor, act as barriers to successful manual scan of retinal tissue. Therefore, a novel sensorless framework is proposed for real-time semi-autonomous endomicroscopy scanning during retinal surgery. The framework consists of the Steady-Hand Eye Robot (SHER) integrated with a pCLE system, where the motion of the probe is controlled semi-autonomously. Through a hybrid motion control strategy, the system autonomously controls the confocal probe to optimize the sharpness and quality of the pCLE images, while providing the surgeon with the ability to scan the tissue in a tremor-free manner. Effectiveness of the proposed architecture is validated through experimental evaluations as well as a user study involving 9 participants. It is shown through statistical analyses that the proposed framework can reduce the work load experienced by the users in a statistically-significant manner, while also enhancing their performance in retaining pCLE images with optimized quality.