Surgical Navigation in the Anterior Skull Base Using 3-Dimensional Endoscopy and Surface Reconstruction.

Importance: Image guidance is an important adjunct for endoscopic sinus and skull base surgery. However, current systems require bulky external tracking equipment, and their use can interrupt efficient surgical workflow. Objective: To evaluate a trackerless surgical navigation system using 3-dimensional (3D) endoscopy and simultaneous localization and mapping (SLAM) algorithms in the anterior skull base. Design, Setting, and Participants: This interventional deceased donor cohort study and retrospective clinical case study was conducted at a tertiary academic medical center with human deceased donor specimens and a patient with anterior skull base pathology. Exposures: Participants underwent endoscopic endonasal transsphenoidal dissection and surface model reconstruction from stereoscopic video with registration to volumetric models segmented from computed tomography (CT) and magnetic resonance imaging. Main Outcomes and Measures: To assess the fidelity of surface model reconstruction and accuracy of surgical navigation and surface-CT model coregistration, 3 metrics were calculated: reconstruction error, registration error, and localization error. Results: In deceased donor models (n = 9), high-fidelity surface models of the posterior wall of the sphenoid sinus were reconstructed from stereoscopic video and coregistered to corresponding volumetric CT models. The mean (SD; range) reconstruction, registration, and localization errors were 0.60 (0.24; 0.36-0.93), 1.11 (0.49; 0.71-1.56) and 1.01 (0.17; 0.78-1.25) mm, respectively. In a clinical case study of a patient who underwent a 3D endoscopic endonasal transsphenoidal resection of a tubercular meningioma, a high-fidelity surface model of the posterior wall of the sphenoid was reconstructed from intraoperative stereoscopic video and coregistered to a volumetric preoperative fused CT magnetic resonance imaging model with a root-mean-square error of 1.38 mm. Conclusions and Relevance: The results of this study suggest that SLAM algorithm-based endoscopic endonasal surgery navigation is a novel, accurate, and trackerless approach to surgical navigation that uses 3D endoscopy and SLAM-based algorithms in lieu of conventional optical or electromagnetic tracking. While multiple challenges remain before clinical readiness, a SLAM algorithm-based endoscopic endonasal surgery navigation system has the potential to improve surgical efficiency, economy of motion, and safety.

Image-guided surgical planning of percutaneous nephrolithotomy with patient-specific CTRs.

PURPOSE: We propose the utilization of patient-specific concentric-tube robots (CTRs) whose designs are optimized to enhance their volumetric reachability of the renal stone, thus reducing the morbidities associated with percutaneous nephrolithotomy procedures. By employing a nested optimization-driven scheme, this work aims to determine a single surgical tract through which the patient-tailored CTR is deployed. We carry out a sensitivity analysis on the combined percutaneous access and optimized CTR design with respect to breathing-induced excursion of the kidneys based on preoperative images. Further, an investigation is also performed of the appropriateness and effectiveness of the percutaneous access provided by the proposed algorithm compared to that of an expert urologist. METHODS: The method is based on an ellipsoidal approximation to the renal calculi and a grid search over candidate skin areas and available renal calyces using an anatomically constrained kinematic mapping of the CTR. Percutaneous access is selected for collision-free CTR deployment to the centroid of the stones with minimal positional error at the renal calyx. Further optimization of the CTR design results in a robot tailored to the therapeutic anatomical features of each clinical case. The study examined 14 sets of clinical data of PCNL patients, analyzing stone reachability using preoperative images and breathing-induced motions of the kidney. An experienced urologist qualitatively assessed the adequacy of percutaneous access generated by the algorithm. RESULTS: An assessment conducted by an expert urologist found that the percutaneous accesses produced by the proposed approach were found to be comparable to those chosen by the expert surgeon in most clinical cases. The simulated results demonstrated a mean volume coverage of [Formula: see text] for static anatomy and [Formula: see text] and [Formula: see text] when considering a 1 cm excursion of the kidney in the craniocaudal directions due to respiration or tool-tissue interaction. CONCLUSION: The optimization-driven scheme for determining a single tract surgical plan, coupled with the use of a patient-specific CTR, shows promising results for improving percutaneous access in PCNL procedures. This approach clearly shows the potential for enhancing the quality and suitability of percutaneous accesses, addressing the challenges posed by staghorn and non-staghorn stones during PCNL procedures. Further research involving clinical validation is necessary to confirm these findings and explore the potential clinical benefits of the approach.

Double-Uncertainty Guided Spatial and Temporal Consistency Regularization Weighting for Learning-based Abdominal Registration.

In order to tackle the difficulty associated with the ill-posed nature of the image registration problem, regularization is often used to constrain the solution space. For most learning-based registration approaches, the regularization usually has a fixed weight and only constrains the spatial transformation. Such convention has two limitations: (i) Besides the laborious grid search for the optimal fixed weight, the regularization strength of a specific image pair should be associated with the content of the images, thus the "one value fits all" training scheme is not ideal; (ii) Only spatially regularizing the transformation may neglect some informative clues related to the ill-posedness. In this study, we propose a mean-teacher based registration framework, which incorporates an additional temporal consistency regularization term by encouraging the teacher model's prediction to be consistent with that of the student model. More importantly, instead of searching for a fixed weight, the teacher enables automatically adjusting the weights of the spatial regularization and the temporal consistency regularization by taking advantage of the transformation uncertainty and appearance uncertainty. Extensive experiments on the challenging abdominal CT-MRI registration show that our training strategy can promisingly advance the original learning-based method in terms of efficient hyperparameter tuning and a better tradeoff between accuracy and smoothness.

On Surgical Planning of Percutaneous Nephrolithotomy with Patient-Specific CTRs.

Percutaneous nephrolithotomy (PCNL) is considered a first-choice minimally invasive procedure for treating kidney stones larger than 2 cm. It yields higher stone-free rates than other minimally invasive techniques and is employed when extracorporeal shock wave lithotripsy or uteroscopy are, for instance, infeasible. Using this technique, surgeons create a tract through which a scope is inserted for gaining access to the stones. Traditional PCNL tools, however, present limited maneuverability, may require multiple punctures and often lead to excessive torquing of the instruments which can damage the kidney parenchyma and thus increase the risk of hemorrhage. We approach this problem by proposing a nested optimization-driven scheme for determining a single tract surgical plan along which a patient-specific concentric-tube robot (CTR) is deployed so as to enhance manipulability along the most dominant directions of the stone presentations. The approach is illustrated with seven sets of clinical data from patients who underwent PCNL. The simulated results may set the stage for achieving higher stone-free rates through single tract PCNL interventions while decreasing blood loss.

UNIMODAL CYCLIC REGULARIZATION FOR TRAINING MULTIMODAL IMAGE REGISTRATION NETWORKS.

The loss function of an unsupervised multimodal image registration framework has two terms, i.e., a metric for similarity measure and regularization. In the deep learning era, researchers proposed many approaches to automatically learn the similarity metric, which has been shown effective in improving registration performance. However, for the regularization term, most existing multimodal registration approaches still use a hand-crafted formula to impose artificial properties on the estimated deformation field. In this work, we propose a unimodal cyclic regularization training pipeline, which learns task-specific prior knowledge from simpler unimodal registration, to constrain the deformation field of multimodal registration. In the experiment of abdominal CT-MR registration, the proposed method yields better results over conventional regularization methods, especially for severely deformed local regions.

UNSUPERVISED MULTIMODAL IMAGE REGISTRATION WITH ADAPTATIVE GRADIENT GUIDANCE.

Multimodal image registration (MIR) is a fundamental procedure in many image-guided therapies. Recently, unsupervised learning-based methods have demonstrated promising performance over accuracy and efficiency in deformable image registration. However, the estimated deformation fields of the existing methods fully rely on the to-be-registered image pair. It is difficult for the networks to be aware of the mismatched boundaries, resulting in unsatisfactory organ boundary alignment. In this paper, we propose a novel multimodal registration framework, which elegantly leverages the deformation fields estimated from both: (i) the original to-be-registered image pair, (ii) their corresponding gradient intensity maps, and adaptively fuses them with the proposed gated fusion module. With the help of auxiliary gradient-space guidance, the network can concentrate more on the spatial relationship of the organ boundary. Experimental results on two clinically acquired CT-MRI datasets demonstrate the effectiveness of our proposed approach.

Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration.

Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Distinct from other translation-based methods that attempt to convert the multimodal problem (e.g., CT-to-MR) into a unimodal problem (e.g., MR-to-MR) via image-to-image translation, our method leverages the deformation fields estimated from both: (i) the translated MR image and (ii) the original CT image in a dual-stream fashion, and automatically learns how to fuse them to achieve better registration performance. The multimodal registration network can be effectively trained by computationally efficient similarity metrics without any ground-truth deformation. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.

Real-Time Dense Reconstruction of Tissue Surface From Stereo Optical Video.

We propose an approach to reconstruct dense three-dimensional (3D) model of tissue surface from stereo optical videos in real-time, the basic idea of which is to first extract 3D information from video frames by using stereo matching, and then to mosaic the reconstructed 3D models. To handle the common low-texture regions on tissue surfaces, we propose effective post-processing steps for the local stereo matching method to enlarge the radius of constraint, which include outliers removal, hole filling, and smoothing. Since the tissue models obtained by stereo matching are limited to the field of view of the imaging modality, we propose a model mosaicking method by using a novel feature-based simultaneously localization and mapping (SLAM) method to align the models. Low-texture regions and the varying illumination condition may lead to a large percentage of feature matching outliers. To solve this problem, we propose several algorithms to improve the robustness of the SLAM, which mainly include 1) a histogram voting-based method to roughly select possible inliers from the feature matching results; 2) a novel 1-point RANSAC-based [Formula: see text] algorithm called as DynamicR1PP [Formula: see text] to track the camera motion; and 3) a GPU-based iterative closest points (ICP) and bundle adjustment (BA) method to refine the camera motion estimation results. Experimental results on ex- and in vivo data showed that the reconstructed 3D models have high-resolution texture with an accuracy error of less than 2 mm. Most algorithms are highly parallelized for GPU computation, and the average runtime for processing one key frame is 76.3 ms on stereo images with 960×540 resolution.

Re-weighting and 1-Point RANSAC-Based P nP Solution to Handle Outliers.

The ability to handle outliers is essential for performing the perspective- n-point (P nP) approach in practical applications, but conventional RANSAC+P3P or P4P methods have high time complexities. We propose a fast P nP solution named R1PP nP to handle outliers by utilizing a soft re-weighting mechanism and the 1-point RANSAC scheme. We first present a P nP algorithm, which serves as the core of R1PP nP, for solving the P nP problem in outlier-free situations. The core algorithm is an optimal process minimizing an objective function conducted with a random control point. Then, to reduce the impact of outliers, we propose a reprojection error-based re-weighting method and integrate it into the core algorithm. Finally, we employ the 1-point RANSAC scheme to try different control points. Experiments with synthetic and real-world data demonstrate that R1PP nP is faster than RANSAC+P3P or P4P methods especially when the percentage of outliers is large, and is accurate. Besides, comparisons with outlier-free synthetic data show that R1PP nP is among the most accurate and fast P nP solutions, which usually serve as the final refinement step of RANSAC+P3P or P4P. Compared with REPP nP, which is the state-of-the-art P nP algorithm with an explicit outliers-handling mechanism, R1PP nP is slower but does not suffer from the percentage of outliers limitation as REPP nP.

Real-Time Surface Deformation Recovery from Stereo Videos.

Tissue deformation during the surgery may significantly decrease the accuracy of surgical navigation systems. In this paper, we propose an approach to estimate the deformation of tissue surface from stereo videos in real-time, which is capable of handling occlusion, smooth surface and fast deformation. We first use a stereo matching method to extract depth information from stereo video frames and generate the tissue template, and then estimate the deformation of the obtained template by minimizing ICP, ORB feature matching and as-rigid-as-possible (ARAP) costs. The main novelties are twofold: (1) Due to non-rigid deformation, feature matching outliers are difficult to be removed by traditional RANSAC methods; therefore we propose a novel 1-point RANSAC and reweighting method to preselect matching inliers, which handles smooth surfaces and fast deformations. (2) We propose a novel ARAP cost function based on dense connections between the control points to achieve better smoothing performance with limited number of iterations. Algorithms are designed and implemented for GPU parallel computing. Experiments on ex- and in vivo data showed that this approach works at an update rate of 15 Hz with an accuracy of less than 2.5 mm on a NVIDIA Titan X GPU.

Increasing the impact of medical image computing using community-based open-access hackathons: The NA-MIC and 3D Slicer experience.

The National Alliance for Medical Image Computing (NA-MIC) was launched in 2004 with the goal of investigating and developing an open source software infrastructure for the extraction of information and knowledge from medical images using computational methods. Several leading research and engineering groups participated in this effort that was funded by the US National Institutes of Health through a variety of infrastructure grants. This effort transformed 3D Slicer from an internal, Boston-based, academic research software application into a professionally maintained, robust, open source platform with an international leadership and developer and user communities. Critical improvements to the widely used underlying open source libraries and tools-VTK, ITK, CMake, CDash, DCMTK-were an additional consequence of this effort. This project has contributed to close to a thousand peer-reviewed publications and a growing portfolio of US and international funded efforts expanding the use of these tools in new medical computing applications every year. In this editorial, we discuss what we believe are gaps in the way medical image computing is pursued today; how a well-executed research platform can enable discovery, innovation and reproducible science ("Open Science"); and how our quest to build such a software platform has evolved into a productive and rewarding social engineering exercise in building an open-access community with a shared vision.

Incremental value of contrast enhanced computed tomography on diagnostic accuracy in evaluation of small pulmonary ground glass nodules.

BACKGROUND: To evaluate the information gain by the application of both non-contrast and contrast enhanced computed tomography (CT) with extended mediastinal display window settings in the evaluation of pure ground glass nodules (pGGNs) and or mixed ground glass nodules (mGGNs) in the context of pre-invasive or early stage lung adenocarcinoma. METHODS: One hundred and fifty patients with ground glass nodules (GGNs) and mGGNs, with contrast enhanced CT scans within 2 weeks of thoracic surgery were included in the study. Quantitative evaluation of all nodules was performed in a conventional mediastinal window (CMW) and an extended mediastinal window (EMW) both on non-contrast images and contrast-enhanced images. RESULTS: Contrast-enhanced images with CMW demonstrated amplification of solid portion in 23 (43%), 41 (77%) with EMW out of 53 minimally invasive adenocarcinoma (MIA) nodules, and in 34 of 37 (91%) of invasive adenocarcinoma (IAC) nodules. Using the increase in size of solid portion of the nodule measured on the enhanced CT images with EMW, area under the receiver operating characteristic (ROC) curve of 0.872 and 0.899 was utilized for differentiating between the pre-invasive nodules and MIA and between MIA and IAC nodules, respectively. Statistically significant differences existed between the pre-invasive and the MIA groups, and MIA and the IAC groups in smaller nodules (P<0.01). CONCLUSIONS: Comparative quantitative analysis of the pre and post contrast images can help differentiate between atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), MIAs, and IACs. Extension of the CT mediastinal window setting improves the evaluation of small GGNs, and can augment the diagnostic accuracy when evaluating small pGGNs and mGGNs.

Magnetic Resonance Imaging-Guided Breast Interventions: Role in Biopsy Targeting and Lumpectomies.

Contrast-enhanced breast MR imaging is increasingly being used to diagnose breast cancer and to perform biopsy procedures. The American Cancer Society has advised women at high risk for breast cancer to have breast MR imaging screening as an adjunct to screening mammography. This article places special emphasis on biopsy and operative planning involving MR imaging and reviews use of breast MR imaging in monitoring response to neoadjuvant chemotherapy. Described are peer-reviewed data on currently accepted MR imaging-guided procedures for addressing benign and malignant breast diseases, including intraoperative imaging.

MRI kinetics with volumetric analysis in correlation with hormonal receptor subtypes and histologic grade of invasive breast cancers.

OBJECTIVE. The aim of this study was to assess whether computer-assisted detection-processed MRI kinetics data can provide further information on the biologic aggressiveness of breast tumors. MATERIALS AND METHODS. We identified 194 newly diagnosed invasive breast cancers presenting as masses on contrast-enhanced MRI by a HIPAA-compliant pathology database search. Computer-assisted detection-derived data for the mean and median peak signal intensity percentage increase, most suspicious kinetic curve patterns, and volumetric analysis of the different kinetic patterns by mean percentage tumor volume were compared against the different hormonal receptor (estrogen-receptor [ER], progesterone-receptor [PR], ERRB2 (HER2/neu), and triple-receptor expressivity) and histologic grade subgroups, which were used as indicators of tumor aggressiveness. RESULTS. The means and medians of the peak signal intensity percentage increase were higher in ER-negative, PR-negative, and triple-negative (all p ≤ 0.001), and grade 3 tumors (p = 0.011). Volumetric analysis showed higher mean percentage volume of rapid initial enhancement in biologically more aggressive ER-negative, PR-negative, and triple-negative tumors compared with ER-positive (64% vs 53.6%, p = 0.013), PR-positive (65.4% vs 52.5%, p = 0.001), and nontriple-negative tumors (65.3% vs 54.6%, p = 0.028), respectively. A higher mean percentage volume of rapid washout component was seen in ERRB2-positive tumors compared with ERRB2-negative tumors (27.5% vs 17.9%, p = 0.020). CONCLUSION. Peak signal intensity percentage increase and volume analysis of the different kinetic patterns of breast tumors showed correlation with hormonal receptor and histologic grade indicators of cancer aggressiveness. Computer-assisted detection-derived MRI kinetics data have the potential to further characterize the aggressiveness of an invasive cancer.

MRI findings of radiation-associated angiosarcoma of the breast (RAS).

PURPOSE: To describe the magnetic resonance imaging (MRI) characteristics of radiation-associated breast angiosarcomas (RAS). MATERIALS AND METHODS: In this Institutional Review board (IRB)-approved retrospective study, 57 women were diagnosed with pathologically confirmed RAS during the study period (January 1999 to May 2013). Seventeen women underwent pretreatment breast MRI (prior to surgical resection or chemotherapy), of which 16 studies were available for review. Imaging features, including all available mammograms, ultrasounds, and breast MRIs, of these patients were evaluated by two radiologists independently and correlated with clinical management and outcomes. RESULTS: The median age of patients at original breast cancer diagnosis was 69.3 years (range 42-84 years), with average time from initial radiation therapy to diagnosis of RAS of 7.3 years (range 5.1-9.5 years). Nine women had mammograms (9/16, 56%) and six had breast ultrasound (US) (6/16, 38%) prior to MRI, which demonstrated nonsuspicious findings in 5/9 mammograms and 3/6 ultrasounds. Four patients had distinct intraparenchymal masses on mammogram and MRI. MRI findings included diffuse T2 high signal skin thickening (16/16, 100%). Nearly half (7/16, 44%) of patients had T2 low signal intensity lesions; all lesions rapidly enhanced on postcontrast T1 -weighted imaging. All women underwent surgical resection, with 8/16 (50%) receiving neoadjuvant chemotherapy. Four women died during the study period. CONCLUSION: Clinical, mammographic, and sonographic findings of RAS are nonspecific and may be occult on conventional breast imaging; MRI findings of RAS include rapidly enhancing dermal and intraparenchymal lesions, some of which are low signal on T2 weighted imaging.

Pilot study to evaluate feasibility of image-guided breast-conserving therapy in the advanced multimodal image-guided operating (AMIGO) suite.

BACKGROUND: The rate of reexcision in breast-conserving surgery remains high, leading to delay in initiation of adjuvant therapy, increased cost, increased complications, and negative psychological impact to the patient.1 (-) 3 We initiated a phase 1 clinical trial to determine the feasibility of the use of intraoperative magnetic resonance imaging (MRI) to assess margins in the advanced multimodal image-guided operating (AMIGO) suite. METHODS: All patients received contrast-enhanced three-dimensional MRI while under general anesthesia in the supine position, followed by standard BCT with or without wire guidance and sentinel node biopsy. Additional margin reexcision was performed of suspicious margins and correlated to final pathology (Fig. 1). Feasibility was assessed via two components: demonstration of safety and sterility and acceptable duration of the operation and imaging; and adequacy of intraoperative MRI imaging for interpretation and its comparison to final pathology. Fig. 1 Schema of AMIGO trial RESULTS: Eight patients (mean age 48.5 years), 4 with stage I breast cancer and 4 with stage II breast cancer, were recruited. All patients underwent successful BCT in the AMIGO suite with no AMIGO-specific complications or break in sterility during surgery. The mean operative time was 113 min (range 93-146 min). CONCLUSIONS: Our experience with AMIGO suggests that it is feasible to use intraoperative MRI imaging to evaluate margin assessment in real time. Further research is required to identify modalities that will lead to a reduction in reexcision in breast cancer therapy.