3D US-CT/MRI registration for percutaneous focal liver tumor ablations.

PURPOSE: US-guided percutaneous focal liver tumor ablations have been considered promising curative treatment techniques. To address cases with invisible or poorly visible tumors, registration of 3D US with CT or MRI is a critical step. By taking advantage of deep learning techniques to efficiently detect representative features in both modalities, we aim to develop a 3D US-CT/MRI registration approach for liver tumor ablations. METHODS: Facilitated by our nnUNet-based 3D US vessel segmentation approach, we propose a coarse-to-fine 3D US-CT/MRI image registration pipeline based on the liver vessel surface and centerlines. Then, phantom, healthy volunteer and patient studies are performed to demonstrate the effectiveness of our proposed registration approach. RESULTS: Our nnUNet-based vessel segmentation model achieved a Dice score of 0.69. In healthy volunteer study, 11 out of 12 3D US-MRI image pairs were successfully registered with an overall centerline distance of 4.03±2.68 mm. Two patient cases achieved target registration errors (TRE) of 4.16 mm and 5.22 mm. CONCLUSION: We proposed a coarse-to-fine 3D US-CT/MRI registration pipeline based on nnUNet vessel segmentation models. Experiments based on healthy volunteers and patient trials demonstrated the effectiveness of our registration workflow. Our code and example data are publicly available in this r epository.

Spatiotemporally dynamic electric fields for brain cancer treatment: anin vitroinvestigation.

Objective. The treatment of glioblastoma (GBM) using low intensity electric fields (∼1 V cm-1) is being investigated using multiple implanted bioelectrodes, which was termed intratumoral modulation therapy (IMT). Previous IMT studies theoretically optimized treatment parameters to maximize coverage with rotating fields, which required experimental investigation. In this study, we employed computer simulations to generate spatiotemporally dynamic electric fields, designed and purpose-built an IMT device forin vitroexperiments, and evaluated the human GBM cellular responses to these fields.Approach. After measuring the electrical conductivity of thein vitroculturing medium, we designed experiments to evaluate the efficacy of various spatiotemporally dynamic fields: (a) different rotating field magnitudes, (b) rotating versus non-rotating fields, (c) 200 kHz versus 10 kHz stimulation, and (d) constructive versus destructive interference. A custom printed circuit board (PCB) was fabricated to enable four-electrode IMT in a 24-well plate. Patient derived GBM cells were treated and analyzed for viability using bioluminescence imaging.Main results. The optimal PCB design had electrodes placed 6.3 mm from the center. Spatiotemporally dynamic IMT fields at magnitudes of 1, 1.5, and 2 V cm-1reduced GBM cell viability to 58%, 37% and 2% of sham controls respectively. Rotating versus non-rotating, and 200 kHz versus 10 kHz fields showed no statistical difference. The rotating configuration yielded a significant reduction (p< 0.01) in cell viability (47 ± 4%) compared to the voltage matched (99 ± 2%) and power matched (66 ± 3%) destructive interference cases.Significance. We found the most important factors in GBM cell susceptibility to IMT are electric field strength and homogeneity. Spatiotemporally dynamic electric fields have been evaluated in this study, where improvements to electric field coverage with lower power consumption and minimal field cancellations has been demonstrated. The impact of this optimized paradigm on cell susceptibility justifies its future use in preclinical and clinical trial investigations.

Planning system for the optimization of electric field delivery using implanted electrodes for brain tumor control.

BACKGROUND: The use of non-ionizing electric fields from low-intensity voltage sources (< 10 V) to control malignant tumor growth is showing increasing potential as a cancer treatment modality. A method of applying these low-intensity electric fields using multiple implanted electrodes within or adjacent to tumor volumes has been termed as intratumoral modulation therapy (IMT). PURPOSE: This study explores advancements in the previously established IMT optimization algorithm, and the development of a custom treatment planning system for patient-specific IMT. The practicality of the treatment planning system is demonstrated by implementing the full optimization pipeline on a brain phantom with robotic electrode implantation, postoperative imaging, and treatment stimulation. METHODS: The integrated planning pipeline in 3D Slicer begins with importing and segmenting patient magnetic resonance images (MRI) or computed tomography (CT) images. The segmentation process is manual, followed by a semi-automatic smoothing step that allows the segmented brain and tumor mesh volumes to be smoothed and simplified by applying selected filters. Electrode trajectories are planned manually on the patient MRI or CT by selecting insertion and tip coordinates for a chosen number of electrodes. The electrode tip positions and stimulation parameters (phase shift and voltage) can then be optimized with the custom semi-automatic IMT optimization algorithm where users can select the prescription electric field, voltage amplitude limit, tissue electrical properties, nearby organs at risk, optimization parameters (electrode tip location, individual contact phase shift and voltage), desired field coverage percent, and field conformity optimization. Tables of optimization results are displayed, and the resulting electric field is visualized as a field-map superimposed on the MR or CT image, with 3D renderings of the brain, tumor, and electrodes. Optimized electrode coordinates are transferred to robotic electrode implantation software to enable planning and subsequent implantation of the electrodes at the desired trajectories. RESULTS: An IMT treatment planning system was developed that incorporates patient-specific MRI or CT, segmentation, volume smoothing, electrode trajectory planning, electrode tip location and stimulation parameter optimization, and results visualization. All previous manual pipeline steps operating on diverse software platforms were coalesced into a single semi-automated 3D Slicer-based user interface. Brain phantom validation of the full system implementation was successful in preoperative planning, robotic electrode implantation, and postoperative treatment planning to adjust stimulation parameters based on actual implant locations. Voltage measurements were obtained in the brain phantom to determine the electrical parameters of the phantom and validate the simulated electric field distribution. CONCLUSIONS: A custom treatment planning and implantation system for IMT has been developed in this study and validated on a phantom brain model, providing an essential step in advancing IMT technology toward future clinical safety and efficacy investigations.

3D US-Based Evaluation and Optimization of Tumor Coverage for US-Guided Percutaneous Liver Thermal Ablation.

Complete tumor coverage by the thermal ablation zone and with a safety margin (5 or 10 mm) is required to achieve the entire tumor eradication in liver tumor ablation procedures. However, 2D ultrasound (US) imaging has limitations in evaluating the tumor coverage by imaging only one or multiple planes, particularly for cases with multiple inserted applicators or irregular tumor shapes. In this paper, we evaluate the intra-procedural tumor coverage using 3D US imaging and investigate whether it can provide clinically needed information. Using data from 14 cases, we employed surface- and volume-based evaluation metrics to provide information on any uncovered tumor region. For cases with incomplete tumor coverage or uneven ablation margin distribution, we also proposed a novel margin uniformity -based approach to provide quantitative applicator adjustment information for optimization of tumor coverage. Both the surface- and volume-based metrics showed that 5 of 14 cases had incomplete tumor coverage according to the estimated ablation zone. After applying our proposed applicator adjustment approach, the simulated results showed that 92.9% (13 of 14) cases achieved 100% tumor coverage and the remaining case can benefit by increasing the ablation time or power. Our proposed method can evaluate the intra-procedural tumor coverage and intuitively provide applicator adjustment information for the physician. Our 3D US-based method is compatible with the constraints of conventional US-guided ablation procedures and can be easily integrated into the clinical workflow.

Toward Fluoro-Free Interventions: Using Radial Intracardiac Ultrasound for Vascular Navigation.

Transcatheter cardiovascular interventions have the advantage of patient safety, reduced surgery time and minimal trauma to the patient's body. Transcathether interventions, which are performed percutaneously, are limited by the lack of direct line of sight with the procedural tools and the patient anatomy. Therefore, such interventional procedures rely heavily on image guidance for navigating toward and delivering therapy at the target site. Vascular navigation via the inferior vena cava, from the groin to the heart, is an imperative part of most transcatheter cardiovascular interventions including heart valve repair surgeries and ablation therapy. Traditionally, the inferior vena cava is navigated using fluoroscopic techniques such as venography and computed tomography venography. These X-ray-based techniques can have detrimental effects on the patient as well as the surgical team, causing increased radiation exposure, leading to risk of cancer, fetal defects and eye cataracts. The use of a heavy lead apron has also been reported to cause back pain and spine issues, thus leading to interventionalist's disc disease. We propose the use of a catheter-based ultrasound augmented with electromagnetic tracking technology to generate a vascular roadmap in real time and perform navigation without harmful radiation. In this pilot study, we used spatially tracked intracardiac echocardiography to reconstruct a vessel from a phantom in a 3-D virtual environment. We illustrate how the proposed ultrasound-based navigation will appear in a virtual environment, by navigating a tracked guidewire within the vessels in the phantom without any radiation-based imaging. The geometric accuracy is assessed using a computed tomography scan of the phantom, with a Dice coefficient of 0.79. The average distance between the surfaces of the two models comes out to be 1.7 ± 1.12 mm.

A Robust Edge-Preserving Stereo Matching Method for Laparoscopic Images.

Stereo matching has become an active area of research in the field of computer vision. In minimally invasive surgery, stereo matching provides depth information to surgeons, with the potential to increase the safety of surgical procedures, particularly those performed laparoscopically. Many stereo matching methods have been reported to perform well for natural images, but for images acquired during a laparoscopic procedure, they are limited by image characteristics including illumination differences, weak texture content, specular highlights, and occlusions. To overcome these limitations, we propose a robust edge-preserving stereo matching method for laparoscopic images, comprising an efficient sparse-dense feature matching step, left and right image illumination equalization, and refined disparity optimization. We validated the proposed method using both benchmark biological phantoms and surgical stereoscopic data. Experimental results illustrated that, in the presence of heavy illumination differences between image pairs, texture and textureless surfaces, specular highlights and occlusions, our proposed approach consistently obtains a more accurate estimate of the disparity map than state-of-the-art stereo matching methods in terms of robustness and boundary preservation.

Ultra-High Field 7-Tesla Magnetic Resonance Imaging and Electroencephalography Findings in Epilepsy.

PURPOSE: Assessment of patients for temporal lobe epilepsy (TLE) surgery requires multimodality input, including EEG recordings to ensure optimal surgical planning. Often EEG demonstrates abnormal foci not detected on 1.5T MRI. Ultra-high field MRI at 7T provides improved resolution of the brain. We investigated the utility of 7T MRI to detect potential anatomical abnormalities associated with EEG changes. METHODS: Ultra-high field data were acquired on a 7T MRI scanner for 13 patients with history of drug resistant TLE who had had EEG telemetry recordings. Qualitative evaluation of 7T imaging for presence of focal abnormalities detected on EEG was performed. Correlation of 7T MRI findings with EEG recordings of focal slowing or interictal epileptic spikes (IEDs), and seizures was performed. RESULTS: Assessment of 7T MRI demonstrated concordance with TLE as determined by the multidisciplinary team in 61.5% of cases (n = 8). Among these, 3 patients exhibited supportive abnormal 7T MRI abnormalities not detected by 1.5T MRI. In patients who underwent surgery, 72.7% had concordant histopathology findings with 7T MRI findings (n = 8). However, qualitative assessment of 7T images revealed focal anatomical abnormalities to account for EEG findings in only 15.4% of patients (n = 2). Other regions that were found to have localized IEDs in addition to the lesional temporal lobe, included the contralateral temporal lobe (n = 5), frontal lobe (n = 3), and parieto-occipital lobe (n = 2). CONCLUSION: Ultra-high field 7T MRI findings show concordance with clinical data. However, 7T MRI did not reveal anatomical findings to account for abnormalities detected by EEG.

Image Guidance in Deep Brain Stimulation Surgery to Treat Parkinson's Disease: A Comprehensive Review.

Deep brain stimulation (DBS) is an effective therapy as an alternative to pharmaceutical treatments for Parkinson's disease (PD). Aside from factors such as instrumentation, treatment plans, and surgical protocols, the success of the procedure depends heavily on the accurate placement of the electrode within the optimal therapeutic targets while avoiding vital structures that can cause surgical complications and adverse neurologic effects. Although specific surgical techniques for DBS can vary, interventional guidance with medical imaging has greatly contributed to the development, outcomes, and safety of the procedure. With rapid development in novel imaging techniques, computational methods, and surgical navigation software, as well as growing insights into the disease and mechanism of action of DBS, modern image guidance is expected to further enhance the capacity and efficacy of the procedure in treating PD. This article surveys the state-of-the-art techniques in image-guided DBS surgery to treat PD, and discusses their benefits and drawbacks, as well as future directions on the topic.

Optimization of multi-electrode implant configurations and programming for the delivery of non-ablative electric fields in intratumoral modulation therapy.

PURPOSE: Application of low intensity electric fields to interfere with tumor growth is being increasingly recognized as a promising new cancer treatment modality. Intratumoral modulation therapy (IMT) is a developing technology that uses multiple electrodes implanted within or adjacent tumor regions to deliver electric fields to treat cancer. In this study, the determination of optimal IMT parameters was cast as a mathematical optimization problem, and electrode configurations, programming, optimization, and maximum treatable tumor size were evaluated in the simplest and easiest to understand spherical tumor model. The establishment of electrode placement and programming rules to maximize electric field tumor coverage designed specifically for IMT is the first step in developing an effective IMT treatment planning system. METHODS: Finite element method electric field computer simulations for tumor models with 2 to 7 implanted electrodes were performed to quantify the electric field over time with various parameters, including number of electrodes (2 to 7), number of contacts per electrode (1 to 3), location within tumor volume, and input waveform with relative phase shift between 0 and 2π radians. Homogeneous tissue specific conductivity and dielectric values were assigned to the spherical tumor and surrounding tissue volume. In order to achieve the goal of covering the tumor volume with a uniform threshold of 1 V/cm electric field, a custom least square objective function was used to maximize the tumor volume covered by 1 V/cm time averaged field, while maximizing the electric field in voxels receiving less than this threshold. An additional term in the objective function was investigated with a weighted tissue sparing term, to minimize the field to surrounding tissues. The positions of the electrodes were also optimized to maximize target coverage with the fewest number of electrodes. The complexity of this optimization problem including its non-convexity, the presence of many local minima, and the computational load associated with these stochastic based optimizations led to the use of a custom pattern search algorithm. Optimization parameters were bounded between 0 and 2π radians for phase shift, and anywhere within the tumor volume for location. The robustness of the pattern search method was then evaluated with 50 random initial parameter values. RESULTS: The optimization algorithm was successfully implemented, and for 2 to 4 electrodes, equally spaced relative phase shifts and electrodes placed equidistant from each other was optimal. For 5 electrodes, up to 2.5 cm diameter tumors with 2.0 V, and 4.1 cm with 4.0 V could be treated with the optimal configuration of a centrally placed electrode and 4 surrounding electrodes. The use of 7 electrodes allow for 3.4 cm diameter coverage at 2.0 V and 5.5 cm at 4.0 V. The evaluation of the optimization method using 50 random initial parameter values found the method to be robust in finding the optimal solution. CONCLUSIONS: This study has established a robust optimization method for temporally optimizing electric field tumor coverage for IMT, with the adaptability to optimize a variety of parameters including geometrical and relative phase shift configurations.

Automatic segmentation of the carotid artery and internal jugular vein from 2D ultrasound images for 3D vascular reconstruction.

PURPOSE: In the context of analyzing neck vascular morphology, this work formulates and compares Mask R-CNN and U-Net-based algorithms to automatically segment the carotid artery (CA) and internal jugular vein (IJV) from transverse neck ultrasound (US). METHODS: US scans of the neck vasculature were collected to produce a dataset of 2439 images and their respective manual segmentations. Fourfold cross-validation was employed to train and evaluate Mask RCNN and U-Net models. The U-Net algorithm includes a post-processing step that selects the largest connected segmentation for each class. A Mask R-CNN-based vascular reconstruction pipeline was validated by performing a surface-to-surface distance comparison between US and CT reconstructions from the same patient. RESULTS: The average CA and IJV Dice scores produced by the Mask R-CNN across the evaluation data from all four sets were [Formula: see text] and [Formula: see text]. The average Dice scores produced by the post-processed U-Net were [Formula: see text] and [Formula: see text], for the CA and IJV, respectively. The reconstruction algorithm utilizing the Mask R-CNN was capable of producing accurate 3D reconstructions with majority of US reconstruction surface points being within 2 mm of the CT equivalent. CONCLUSIONS: On average, the Mask R-CNN produced more accurate vascular segmentations compared to U-Net. The Mask R-CNN models were used to produce 3D reconstructed vasculature with a similar accuracy to that of a manually segmented CT scan. This implementation of the Mask R-CNN network enables automatic analysis of the neck vasculature and facilitates 3D vascular reconstruction.

A simple, realistic walled phantom for intravascular and intracardiac applications.

PURPOSE: This work aims to develop a simple, anatomically and haptically realistic vascular phantom, compatible with intravascular and intracardiac ultrasound. The low-cost, dual-layered phantom bridges the gap between traditional wall-only and wall-less phantoms by showing both the vessel wall and surrounding tissue in ultrasound imaging. This phantom can better assist clinical tool training, testing of intravascular devices, blood flow studies, and validation of algorithms for intravascular and intracardiac surgical systems. METHODS: Polyvinyl alcohol cryogel (PVA-c) incorporating a scattering agent was used to obtain vessel and tissue-mimicking materials. Our specific design targeted the inferior vena cava and renal bifurcations which were modelled using CAD software. A custom mould and container were 3D-printed for shaping the desired vessel wall. Three phantoms were prepared by varying both the concentrations of scattering agent as well as the number of freeze-thaw cycles to which the phantom layers were subjected during the manufacturing process. Each phantom was evaluated using ultrasound imaging using the Foresight™ ICE probe. Geometrical validation was provided by comparing CAD design to a CT scan of the phantom. RESULTS: The desired vascular phantom was constructed using 2.5% and 0.05% scattering agent concentration in the vessel and tissue-mimicking layers, respectively. Imaging of the three phantoms showed that increasing the number of freeze-thaw cycles did not significantly enhance the image contrast. Comparison of the US images with their CT equivalents resulted in an average error of 0.9[Formula: see text] for the lumen diameter. CONCLUSION: The phantom is anatomically realistic when imaged with intracardiac ultrasound and provides a smooth lumen for the ultrasound probe and catheter to manoeuvre. The vascular phantom enables validation of intravascular and intracardiac image guidance systems. The simple construction technique also provides a workflow for designing complex, multi-layered arterial phantoms.

Augmented reality simulator for ultrasound-guided percutaneous renal access.

PURPOSE: Traditional training for percutaneous renal access (PCA) relies on apprenticeship, which raises concerns about patient safety, limited training opportunities, and inconsistent quality of feedback. In this study, we proposed the development of a novel augmented reality (AR) simulator for ultrasound (US)-guided PCA and evaluated its validity and efficacy as a teaching tool. METHODS: Our AR simulator allows the user to practice PCA on a silicone phantom using a tracked needle and US probe emulator under the guidance of simulated US on a tablet screen. 6 Expert and 24 novice participants were recruited to evaluate the efficacy of our simulator. RESULTS: Experts highly rated the realism and usefulness of our simulator, reflected by the average face validity score of 4.39 and content validity score of 4.53 on a 5-point Likert scale. Comparisons with a Mann-Whitney U test revealed significant differences [Formula: see text] in performances between the experts and novices on 6 out of 7 evaluation metrics, demonstrating strong construct validity. Furthermore, a paired T-test indicated significant performance improvements [Formula: see text] of the novices in both objective and subjective evaluation after training with our simulator. CONCLUSION: Our cost-effective, flexible, and easily customizable AR training simulator can provide opportunities for trainees to acquire basic skills of US-guided PCA in a safe and stress-free environment. The effectiveness of our simulator is demonstrated through strong face, content, and construct validity, indicating its value as a novel training tool.

Accuracy assessment for the co-registration between optical and VIVE head-mounted display tracking.

PURPOSE: We report on the development and accuracy assessment of a hybrid tracking system that integrates optical spatial tracking into a video pass-through head-mounted display. METHODS: The hybrid system uses a dual-tracked co-calibration apparatus to provide a co-registration between the origins of an optical dynamic reference frame and the VIVE Pro controller through a point-based registration. This registration provides the location of optically tracked tools with respect to the VIVE controller's origin and thus the VIVE's tracking system. RESULTS: The positional accuracy was assessed using a CNC machine to collect a grid of points with 25 samples per location. The positional trueness and precision for the hybrid tracking system were [Formula: see text] and [Formula: see text], respectively. The rotational accuracy was assessed through inserting a stylus tracked by all three systems into a hemispherical phantom with cylindrical openings at known angles and collecting 25 samples per cylinder for each system. The rotational trueness and precision for the hybrid tracking system were [Formula: see text] and [Formula: see text], respectively. The difference in position and rotational trueness between the OTS and the hybrid tracking system was [Formula: see text] and [Formula: see text], respectively. CONCLUSIONS: We developed a hybrid tracking system that allows the pose of optically tracked surgical instruments to be known within a first-person HMD visualization system, achieving submillimeter accuracy. This research validated the positional and rotational accuracy of the hybrid tracking system and subsequently the optical tracking and VIVE tracking systems. This work provides a method to determine the position of an optically tracked surgical tool with a surgically acceptable accuracy within a low-cost commercial-grade video pass-through HMD. The hybrid tracking system provides the foundation for the continued development of virtual reality or augmented virtuality surgical navigation systems for training or practicing surgical techniques.

Endoscopic Vision Augmentation Using Multiscale Bilateral-Weighted Retinex for Robotic Surgery.

Endoscopic vision plays a significant role in minimally invasive surgical procedures. The visibility and maintenance of such direct in situ vision is paramount not only for safety by preventing inadvertent injury but also to improve precision and reduce operating time. Unfortunately, the endoscopic vision is unavoidably degraded due to the illumination variations during surgery. This paper aims to restore or augment such degraded visualization and quantitatively evaluate it during robotic surgery. A multiscale bilateral-weighted retinex method is proposed to remove non-uniform and highly directional illumination and enhance surgical vision, while an objective no-reference image visibility assessment method is defined in terms of sharpness, naturalness, and contrast, to quantitatively and objectively evaluate the endoscopic visualization on surgical video sequences. The methods were validated on surgical data, with the experimental results showing that our method outperforms existent retinex approaches. In particular, the combined visibility was improved from 0.81 to 1.06, while three surgeons generally agreed that the results were restored with much better visibility.

Dynamic, patient-specific mitral valve modelling for planning transcatheter repairs.

PURPOSE: Transcatheter, beating heart repair techniques for mitral valve regurgitation is a very active area of development. However, it is difficult to both simulate and predict the clinical outcomes of mitral repairs, owing to the complexity of mitral valve geometry and the influence of hemodynamics. We aim to produce a workflow for manufacturing dynamic patient-specific models to simulate the mitral valve for transcatheter repair applications. METHODS: In this paper, we present technology and associated workflow, for using transesophageal echocardiography to generate dynamic physical replicas of patient valves. We validate our workflow using six patient datasets representing patients with unique or particularly challenging pathologies as selected by a cardiologist. The dynamic component of the models and their resultant potential as procedure planning tools is due to a dynamic pulse duplicator that permits the evaluation of the valve models experiencing realistic hemodynamics. RESULTS: Early results indicate the workflow has excellent anatomical accuracy and the ability to replicate regurgitation pathologies, as shown by colour Doppler ultrasound and anatomical measurements comparing patients and models. Analysis of all measurements successfully resulted in t critical two-tail > t stat and p values > 0.05, thus demonstrating no statistical difference between the patients and models, owing to high fidelity morphological replication. CONCLUSIONS: Due to the combination of a dynamic environment and patient-specific modelling, this workflow demonstrates a promising technology for simulating the complete morphology of mitral valves undergoing transcatheter repairs.

Robust, Intrinsic Tracking of a Laparoscopic Ultrasound Probe for Ultrasound-Augmented Laparoscopy.

In situ visualization of laparoscopic ultrasound in both conventional and robot-assisted laparoscopic surgery requires robust and efficient computation of the pose of the laparoscopic ultrasound probe with respect to the laparoscopic camera. Image-based intrinsic methods of computing this relative pose need to overcome challenges due to irregular illumination, partial feature occlusion, and clutter that are unavoidable in practical laparoscopic surgery. In this paper, we propose an accurate image-based method that is robust to partial occlusion of the fiducials and outliers. The method is extended to multi-view imaging model with applications in stereoscopic laparoscopy and robot-assisted surgery. Rather than treating the model-to-image correspondence and pose computation as separate problems, we solve them jointly using the Kalman Filter-based framework that demonstrates video rate running time (~24fps). By keeping the optical tracking measurements as a reference, we demonstrate that the proposed methods result in clinically acceptable tracking accuracy, reaching target registration errors well below 1.5mm on average. In addition, our multi-view tracking method is compared to a conventional stereo triangulation-based pose estimation scheme that commercial optical tracking systems are based on, to experimentally demonstrate its superiority in terms of accuracy. Finally, we qualitatively demonstrate the suitability of our methods for practical laparoscopic applications by conducting a phantom-based experiment.

Advanced Endoscopic Navigation: Surgical Big Data, Methodology, and Applications.

Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.

Dynamic Patient-Specific Three-Dimensional Simulation of Mitral Repair: Can We Practice Mitral Repair Preoperatively?

OBJECTIVE: Planned mitral repair strategies are generally established from preoperative echocardiography; however, specific details of the repair are often determined intraoperatively. We propose that three-dimensional printed, patient-specific, dynamic mitral valve models may help surgeons plan and trial all the details of a specific patient's mitral repair preoperatively. METHODS: Using preoperative echocardiography, segmentation, modeling software, and three-dimensional printing, we created dynamic, high-fidelity, patient-specific mitral valve models including the subvalvular apparatus. We assessed the accuracy of 10 patient mitral valve models anatomically and functionally in a heart phantom simulator, both objectively by blinded echocardiographic assessment, and subjectively by two mitral repair experts. After this, we attempted model mitral repair and compared the outcomes with postoperative echocardiography. RESULTS: Model measurements were accurate when compared with patients on anterior-posterior diameter, circumference, and anterior leaflet length; however, less accurate on posterior leaflet length. On subjective assessment, Likert scores were high at 3.8 ± 0.4 and 3.4 ± 0.7, suggesting good fidelity of the dynamic model echocardiogram and functional model in the phantom to the preoperative three-dimensional echocardiogram, respectively. Mitral repair was successful in all 10 models with significant reduction in mitral insufficiency. In two models, mitral repair was performed twice, using two different surgical techniques to assess which provided a better outcome. When compared with the actual patient mitral repair outcome, the repaired models compared favorably. CONCLUSIONS: Complex mitral valve modeling seems to predict an individual patient's mitral anatomy well, before surgery. Further investigation is required to determine whether deliberate preoperative practice can improve mitral repair outcomes.

Mixed reality ultrasound guidance system: a case study in system development and a cautionary tale.

PURPOSE: Real-time ultrasound has become a crucial aspect of several image-guided interventions. One of the main constraints of such an approach is the difficulty in interpretability of the limited field of view of the image, a problem that has recently been addressed using mixed reality, such as augmented reality and augmented virtuality. The growing popularity and maturity of mixed reality has led to a series of informal guidelines to direct development of new systems and to facilitate regulatory approval. However, the goals of mixed reality image guidance systems and the guidelines for their development have not been thoroughly discussed. The purpose of this paper is to identify and critically examine development guidelines in the context of a mixed reality ultrasound guidance system through a case study. METHODS: A mixed reality ultrasound guidance system tailored to central line insertions was developed in close collaboration with an expert user. This system outperformed ultrasound-only guidance in a novice user study and has obtained clearance for clinical use in humans. A phantom study with 25 experienced physicians was carried out to compare the performance of the mixed reality ultrasound system against conventional ultrasound-only guidance. Despite the previous promising results, there was no statistically significant difference between the two systems. RESULTS: Guidelines for developing mixed reality image guidance systems cannot be applied indiscriminately. Each design decision, no matter how well justified, should be the subject of scientific and technical investigation. Iterative and small-scale evaluation can readily unearth issues and previously unknown or implicit system requirements. CONCLUSIONS: We recommend a wary eye in development of mixed reality ultrasound image guidance systems emphasizing small-scale iterative evaluation alongside system development. Ultimately, we recommend that the image-guided intervention community furthers and deepens this discussion into best practices in developing image-guided interventions.