Multimodal MRI segmentation of key structures for microvascular decompression via knowledge-driven mutual distillation and topological constraints.

PURPOSE: Microvascular decompression (MVD) is a widely used neurosurgical intervention for the treatment of cranial nerves compression. Segmentation of MVD-related structures, including the brainstem, nerves, arteries, and veins, is critical for preoperative planning and intraoperative decision-making. Automatically segmenting structures related to MVD is still challenging for current methods due to the limited information from a single modality and the complex topology of vessels and nerves. METHODS: Considering that it is hard to distinguish MVD-related structures, especially for nerve and vessels with similar topology, we design a multimodal segmentation network with a shared encoder-dual decoder structure and propose a clinical knowledge-driven distillation scheme, allowing reliable knowledge transferred from each decoder to the other. Besides, we introduce a class-wise contrastive module to learn the discriminative representations by maximizing the distance among classes across modalities. Then, a projected topological loss based on persistent homology is proposed to constrain topological continuity. RESULTS: We evaluate the performance of our method on in-house dataset consisting of 100 paired HR-T2WI and 3D TOF-MRA volumes. Experiments indicate that our model outperforms the SOTA in DSC by 1.9% for artery, 3.3% for vein and 0.5% for nerve. Visualization results show our method attains improved continuity and less breakage, which is also consistent with intraoperative images. CONCLUSION: Our method can comprehensively extract the distinct features from multimodal data to segment the MVD-related key structures and preserve the topological continuity, allowing surgeons precisely perceiving the patient-specific target anatomy and substantially reducing the workload of surgeons in the preoperative planning stage. Our resources will be publicly available at https://github.com/JaronTu/Multimodal_MVD_Seg .

Flow diverters treatment planning of small- and medium-sized intracranial saccular aneurysms on the internal carotid artery via constraint-based virtual deployment.

PURPOSE: The internal carotid artery (ICA) is a region with a high incidence for small- and medium-sized saccular aneurysms. However, the treatment relies heavily on the surgeon's experience to achieve optimal outcome. Although the finite element method (FEM) and computational fluid dynamics can predict the postoperative outcomes, due to the computational complexity of traditional methods, there is an urgent need for investigating the fast but versatile approaches related to numerical simulations of flow diverters (FDs) deployment coupled with the hemodynamic analysis to determine the treatment plan. METHODS: We collected the preoperative and postoperative data from 34 patients (29 females, 5 males; mean age 55.74 ± 9.98 years) who were treated with a single flow diverter for small- to medium-sized intracranial saccular aneurysms on the ICA. The constraint-based virtual deployment (CVD) method is proposed to simulate the FDs expanding outward along the vessel centerline while be constrained by the inner wall of the vessel. RESULTS: The results indicate that there were no significant differences in the reduction rates of wall shear stress and aneurysms neck velocity between the FEM and methods. However, the solution time of CVD was greatly reduced by 98%. CONCLUSION: In the typical location of small- and medium-sized saccular aneurysms, namely the ICA, our virtual FDs deployment simulation effectively balances the computational accuracy and efficiency. Combined with hemodynamics analysis, our method can accurately represent the blood flow changes within the lesion region to assist surgeons in clinical decision-making.

Risk prediction and analysis of gallbladder polyps with deep neural network.

The aim of this study is to analyze the risk factors associated with the development of adenomatous and malignant polyps in the gallbladder. Adenomatous polyps of the gallbladder are considered precancerous and have a high likelihood of progressing into malignancy. Preoperatively, distinguishing between benign gallbladder polyps, adenomatous polyps, and malignant polyps is challenging. Therefore, the objective is to develop a neural network model that utilizes these risk factors to accurately predict the nature of polyps. This predictive model can be employed to differentiate the nature of polyps before surgery, enhancing diagnostic accuracy. A retrospective study was done on patients who had cholecystectomy surgeries at the Department of Hepatobiliary Surgery of the Second People's Hospital of Shenzhen between January 2017 and December 2022. The patients' clinical characteristics, lab results, and ultrasonographic indices were examined. Using risk variables for the growth of adenomatous and malignant polyps in the gallbladder, a neural network model for predicting the kind of polyps will be created. A normalized confusion matrix, PR, and ROC curve were used to evaluate the performance of the model. In this comprehensive study, we meticulously analyzed a total of 287 cases of benign gallbladder polyps, 15 cases of adenomatous polyps, and 27 cases of malignant polyps. The data analysis revealed several significant findings. Specifically, hepatitis B core antibody (95% CI -0.237 to 0.061, p < 0.001), number of polyps (95% CI -0.214 to -0.052, p = 0.001), polyp size (95% CI 0.038 to 0.051, p < 0.001), wall thickness (95% CI 0.042 to 0.081, p < 0.001), and gallbladder size (95% CI 0.185 to 0.367, p < 0.001) emerged as independent predictors for gallbladder adenomatous polyps and malignant polyps. Based on these significant findings, we developed a predictive classification model for gallbladder polyps, represented as follows, Predictive classification model for GBPs = -0.149 * core antibody - 0.033 * number of polyps + 0.045 * polyp size + 0.061 * wall thickness + 0.276 * gallbladder size - 4.313. To assess the predictive efficiency of the model, we employed precision-recall (PR) and receiver operating characteristic (ROC) curves. The area under the curve (AUC) for the prediction model was 0.945 and 0.930, respectively, indicating excellent predictive capability. We determined that a polyp size of 10 mm served as the optimal cutoff value for diagnosing gallbladder adenoma, with a sensitivity of 81.5% and specificity of 60.0%. For the diagnosis of gallbladder cancer, the sensitivity and specificity were 81.5% and 92.5%, respectively. These findings highlight the potential of our predictive model and provide valuable insights into accurate diagnosis and risk assessment for gallbladder polyps. We identified several risk factors associated with the development of adenomatous and malignant polyps in the gallbladder, including hepatitis B core antibodies, polyp number, polyp size, wall thickness, and gallbladder size. To address the need for accurate prediction, we introduced a novel neural network learning algorithm. This algorithm utilizes the aforementioned risk factors to predict the nature of gallbladder polyps. By accurately identifying the nature of these polyps, our model can assist patients in making informed decisions regarding their treatment and management strategies. This innovative approach aims to improve patient outcomes and enhance the overall effectiveness of care.

Enhancing the depth perception of DSA images with 2D-3D registration.

Objective: Today, cerebrovascular disease has become an important health hazard. Therefore, it is necessary to perform a more accurate and less time-consuming registration of preoperative three-dimensional (3D) images and intraoperative two-dimensional (2D) projection images which is very important for conducting cerebrovascular disease interventions. The 2D-3D registration method proposed in this study is designed to solve the problems of long registration time and large registration errors in 3D computed tomography angiography (CTA) images and 2D digital subtraction angiography (DSA) images. Methods: To make a more comprehensive and active diagnosis, treatment and surgery plan for patients with cerebrovascular diseases, we propose a weighted similarity measure function, the normalized mutual information-gradient difference (NMG), which can evaluate the 2D-3D registration results. Then, using a multi-resolution fusion optimization strategy, the multi-resolution fused regular step gradient descent optimization (MR-RSGD) method is presented to attain the optimal value of the registration results in the process of the optimization algorithm. Result: In this study, we adopt two datasets of the brain vessels to validate and obtain similarity metric values which are 0.0037 and 0.0003, respectively. Using the registration method proposed in this study, the time taken for the experiment was calculated to be 56.55s and 50.8070s, respectively, for the two sets of data. The results show that the registration methods proposed in this study are both better than the Normalized Mutual (NM) and Normalized Mutual Information (NMI). Conclusion: The experimental results in this study show that in the 2D-3D registration process, to evaluate the registration results more accurately, we can use the similarity metric function containing the image gray information and spatial information. To improve the efficiency of the registration process, we can choose the algorithm with gradient optimization strategy. Our method has great potential to be applied in practical interventional treatment for intuitive 3D navigation.

Versatile multi-constrained planning for thermal ablation of large liver tumors.

The surgical planning of large hepatic tumor ablation remains a challenging task that relies on fulfilling multiple medical constraints, especially for the ablation based on configurations of multiple electrodes. The placement of the electrodes to completely ablate the tumor as well as their insertion trajectory to their final position have to be planned to cause as little damage to healthy anatomical structures as possible to allow a fast rehabilitation. In this paper, we present a novel, versatile approach for the computer-assisted planning of multi-electrode thermal ablation of large liver tumors based on pre-operative CT data with semantic annotations. This involves both the specification of the number of required electrodes and their distribution to adequately ablate the tumor region without damaging too much healthy tissue. To determine the insertion trajectory of the electrodes to their final position, we additionally incorporate a series of medical constraints into our optimization, which allows a global analysis where obstacles such as bones are taken into account and damage to healthy tissue is mitigated. Compared with the state-of-the-art method, our method achieves compact ablation regions without relying on assumptions on a potential needle path for optimal global search and, hence, is suitable for guiding clinicians through the planning of the tumor ablation. We also demonstrate the feasibility of our approach in various experiments of clinical data and demonstrate that our approach not only allows completely ablating the tumor region but also reducing the damage of healthy tissue in comparison to the previous state-of-the-art method.

Towards quantitative and intuitive percutaneous tumor puncture via augmented virtual reality.

In recent years, the radiofrequency ablation (RFA) therapy has become a widely accepted minimal invasive treatment for liver tumor patients. However, it is challenging for doctors to precisely and efficiently perform the percutaneous tumor punctures under free-breathing conditions. This is because the traditional RFA is based on the 2D CT Image information, the missing spatial and dynamic information is dependent on surgeons' experience. This paper presents a novel quantitative and intuitive surgical navigation modality for percutaneous respiratory tumor puncture via augmented virtual reality, which is to achieve the augmented visualization of the pre-operative virtual planning information precisely being overlaid on intra-operative surgical scenario. In the pre-operation stage, we first combine the signed distance field of feasible structures (like liver and tumor) where the puncture path can go through and unfeasible structures (like large vessels and ribs) where the needle is not allowed to go through to quantitatively generate the 3D feasible region for percutaneous puncture. Then we design three constraints according to the RFA specialists consensus to automatically determine the optimal puncture trajectory. In the intra-operative stage, we first propose a virtual-real alignment method to precisely superimpose the virtual information on surgical scenario. Then, a user-friendly collaborative holographic interface is designed for real-time 3D respiratory tumor puncture navigation, which can effectively assist surgeons fast and accurately locating the target step-by step. The validation of our system is performed on static abdominal phantom and in vivo beagle dogs with artificial lesion. Experimental results demonstrate that the accuracy of the proposed planning strategy is better than the manual planning sketched by experienced doctors. Besides, the proposed holographic navigation modality can effectively reduce the needle adjustment for precise puncture as well. Our system shows its clinical feasibility to provide the quantitative planning of optimal needle path and intuitive in situ holographic navigation for percutaneous tumor ablation without surgeons' experience-dependence and reduce the times of needle adjustment. The proposed augmented virtual reality navigation system can effectively improve the precision and reliability in percutaneous tumor ablation and has the potential to be used for other surgical navigation tasks.

Automatic identification of sweet spots from MERs for electrodes implantation in STN-DBS.

PURPOSE: Microelectrode recordings (MERs) are a significant clinical indicator for sweet spots identification of implanted electrodes during deep brain stimulation of the subthalamic nucleus (STN) surgery. As 1D MERs signals have the unboundedness, large-range, large-amount and time-dependent characteristics, the purpose of this study is to propose an automatic and precise identification method of sweet spots from MERs, reducing the time-consuming and labor-intensive human annotations. METHODS: We propose an automatic identification method of sweet spots from MERs for electrodes implantation in STN-DBS. To better imitate the surgeons' observation and obtain more intuitive contextual information, we first employ the 2D Gramian angular summation field (GASF) images generated from MERs data to perform the sweet spots determination for electrodes implantation. Then, we introduce the convolutional block attention module into convolutional neural network (CNN) to identify the 2D GASF images of sweet spots for electrodes implantation. RESULTS: Experimental results illustrate that the identification result of our method is consistent with the result of doctor's decision, while our method can achieve the accuracy and precision of 96.72% and 98.97%, respectively, which outperforms state-of-the-art for intraoperative sweet spots determination. CONCLUSIONS: The proposed method is the first time to automatically and accurately identify sweet spots from MERs for electrodes implantation by the combination an advanced time series-to-image encoding way with CBAM-enhanced networks model. Our method can assist neurosurgeons in automatically detecting the most likely locations of sweet spots for electrodes implantation, which can provide an important indicator for target selection while it reduces the localization error of the target during STN-DBS surgery.

Assessing performance of augmented reality-based neurosurgical training.

This paper presents a novel augmented reality (AR)-based neurosurgical training simulator which provides a very natural way for surgeons to learn neurosurgical skills. Surgical simulation with bimanual haptic interaction is integrated in this work to provide a simulated environment for users to achieve holographic guidance for pre-operative training. To achieve the AR guidance, the simulator should precisely overlay the 3D anatomical information of the hidden target organs in the patients in real surgery. In this regard, the patient-specific anatomy structures are reconstructed from segmented brain magnetic resonance imaging. We propose a registration method for precise mapping of the virtual and real information. In addition, the simulator provides bimanual haptic interaction in a holographic environment to mimic real brain tumor resection. In this study, we conduct AR-based guidance validation and a user study on the developed simulator, which demonstrate the high accuracy of our AR-based neurosurgery simulator, as well as the AR guidance mode's potential to improve neurosurgery by simplifying the operation, reducing the difficulty of the operation, shortening the operation time, and increasing the precision of the operation.

Personalized heterogeneous deformable model for fast volumetric registration.

BACKGROUND: Biomechanical deformable volumetric registration can help improve safety of surgical interventions by ensuring the operations are extremely precise. However, this technique has been limited by the accuracy and the computational efficiency of patient-specific modeling. METHODS: This study presents a tissue-tissue coupling strategy based on penalty method to model the heterogeneous behavior of deformable body, and estimate the personalized tissue-tissue coupling parameters in a data-driven way. Moreover, considering that the computational efficiency of biomechanical model is highly dependent on the mechanical resolution, a practical coarse-to-fine scheme is proposed to increase runtime efficiency. Particularly, a detail enrichment database is established in an offline fashion to represent the mapping relationship between the deformation results of high-resolution hexahedral mesh extracted from the raw medical data and a newly constructed low-resolution hexahedral mesh. At runtime, the mechanical behavior of human organ under interactions is simulated with this low-resolution hexahedral mesh, then the microstructures are synthesized in virtue of the detail enrichment database. RESULTS: The proposed method is validated by volumetric registration in an abdominal phantom compression experiments. Our personalized heterogeneous deformable model can well describe the coupling effects between different tissues of the phantom. Compared with high-resolution heterogeneous deformable model, the low-resolution deformable model with our detail enrichment database can achieve 9.4× faster, and the average target registration error is 3.42 mm, which demonstrates that the proposed method shows better volumetric registration performance than state-of-the-art. CONCLUSIONS: Our framework can well balance the precision and efficiency, and has great potential to be adopted in the practical augmented reality image-guided robotic systems.

Constrained Point-Based Framework with Efficient Mechanical Interaction for Virtual Surgery.

This paper proposes a stable and volume conserving meshless approach for soft tissue deformation in virtual surgery, where an efficient tissue-tissue interaction scheme is designed for interacting deformable bodies. Specifically, we integrate position-based volume constraint into point-based framework in order to conserve the total volume of deformable bodies and maintain the stability of the simulation. Moreover, we resolve the tissue-tissue interactions of deformable bodies with position-based contact model, which directly computes the displacement caused by normal and shear stress. The proposed approach can be regarded as a fast approximation of the precise contact model. Experimental results demonstrate that our approach can well enforce the volume conservation of deformable bodies, and obtain visual plausible behaviors for multiple organs interactions.

Modeling and predicting tissue movement and deformation for high intensity focused ultrasound therapy.

PURPOSE: In ultrasound-guided High Intensity Focused Ultrasound (HIFU) therapy, the target tissue (such as a tumor) often moves and/or deforms in response to an external force. This problem creates difficulties in treating patients and can lead to the destruction of normal tissue. In order to solve this problem, we present a novel method to model and predict the movement and deformation of the target tissue during ultrasound-guided HIFU therapy. METHODS: Our method computationally predicts the position of the target tissue under external force. This prediction allows appropriate adjustments in the focal region during the application of HIFU so that the treatment head is kept aligned with the diseased tissue through the course of therapy. To accomplish this goal, we utilize the cow tissue as the experimental target tissue to collect spatial sequences of ultrasound images using the HIFU equipment. A Geodesic Localized Chan-Vese (GLCV) model is developed to segment the target tissue images. A 3D target tissue model is built based on the segmented results. A versatile particle framework is constructed based on Smoothed Particle Hydrodynamics (SPH) to model the movement and deformation of the target tissue. Further, an iterative parameter estimation algorithm is utilized to determine the essential parameters of the versatile particle framework. Finally, the versatile particle framework with the determined parameters is used to estimate the movement and deformation of the target tissue. RESULTS: To validate our method, we compare the predicted contours with the ground truth contours. We found that the lowest, highest and average Dice Similarity Coefficient (DSC) values between predicted and ground truth contours were, respectively, 0.9615, 0.9770 and 0.9697. CONCLUSION: Our experimental result indicates that the proposed method can effectively predict the dynamic contours of the moving and deforming tissue during ultrasound-guided HIFU therapy.