Evaluation of Direct Haptic 4D Volume Rendering of Partially Segmented Data for Liver Puncture Simulation.

This work presents an evaluation study using a force feedback evaluation framework for a novel direct needle force volume rendering concept in the context of liver puncture simulation. PTC/PTCD puncture interventions targeting the bile ducts have been selected to illustrate this concept. The haptic algorithms of the simulator system are based on (1) partially segmented patient image data and (2) a non-linear spring model effective at organ borders. The primary aim is to quantitatively evaluate force errors caused by our patient modeling approach, in comparison to haptic force output obtained from using gold-standard, completely manually-segmented data. The evaluation of the force algorithms compared to a force output from fully manually segmented gold-standard patient models, yields a low mean of 0.12 N root mean squared force error and up to 1.6 N for systematic maximum absolute errors. Force errors were evaluated on 31,222 preplanned test paths from 10 patients. Only twelve percent of the emitted forces along these paths were affected by errors. This is the first study evaluating haptic algorithms with deformable virtual patients in silico. We prove haptic rendering plausibility on a very high number of test paths. Important errors are below just noticeable differences for the hand-arm system.

Efficient patient modeling for visuo-haptic VR simulation using a generic patient atlas.

BACKGROUND AND OBJECTIVE: This work presents a new time-saving virtual patient modeling system by way of example for an existing visuo-haptic training and planning virtual reality (VR) system for percutaneous transhepatic cholangio-drainage (PTCD). METHODS: Our modeling process is based on a generic patient atlas to start with. It is defined by organ-specific optimized models, method modules and parameters, i.e. mainly individual segmentation masks, transfer functions to fill the gaps between the masks and intensity image data. In this contribution, we show how generic patient atlases can be generalized to new patient data. The methodology consists of patient-specific, locally-adaptive transfer functions and dedicated modeling methods such as multi-atlas segmentation, vessel filtering and spline-modeling. RESULTS: Our full image volume segmentation algorithm yields median DICE coefficients of 0.98, 0.93, 0.82, 0.74, 0.51 and 0.48 regarding soft-tissue, liver, bone, skin, blood and bile vessels for ten test patients and three selected reference patients. Compared to standard slice-wise manual contouring time saving is remarkable. CONCLUSIONS: Our segmentation process shows out efficiency and robustness for upper abdominal puncture simulation systems. This marks a significant step toward establishing patient-specific training and hands-on planning systems in a clinical environment.

Real-Time Ultrasound Simulation for Training of US-Guided Needle Insertion in Breathing Virtual Patients.

One draw-back of most existing VR ultrasound training simulators is the use of static 3D patient models neglecting physiological changes induced e.g. by respiration or heart motion. In this paper to the aim of more realistic Ultrasound simulation, breathing motion extracted from 4D CT image data is integrated into our visuo-haptic simulation framework. The simulated ultrasound images are used for the training of US-guided needle insertion procedures in liver surgery. The methodology developed enables US simulation, 3D visualization and haptic steering of the ultrasound probe and the needle in real-time in breathing virtual bodies.

A Virtual Reality System for PTCD Simulation Using Direct Visuo-Haptic Rendering of Partially Segmented Image Data.

This study presents a new visuo-haptic virtual reality (VR) training and planning system for percutaneous transhepatic cholangio-drainage (PTCD) based on partially segmented virtual patient models. We only use partially segmented image data instead of a full segmentation and circumvent the necessity of surface or volume mesh models. Haptic interaction with the virtual patient during virtual palpation, ultrasound probing and needle insertion is provided. Furthermore, the VR simulator includes X-ray and ultrasound simulation for image-guided training. The visualization techniques are GPU-accelerated by implementation in Cuda and include real-time volume deformations computed on the grid of the image data. Computation on the image grid enables straightforward integration of the deformed image data into the visualization components. To provide shorter rendering times, the performance of the volume deformation algorithm is improved by a multigrid approach. To evaluate the VR training system, a user evaluation has been performed and deformation algorithms are analyzed in terms of convergence speed with respect to a fully converged solution. The user evaluation shows positive results with increased user confidence after a training session. It is shown that using partially segmented patient data and direct volume rendering is suitable for the simulation of needle insertion procedures such as PTCD.

Direct Visuo-Haptic 4D Volume Rendering Using Respiratory Motion Models.

This article presents methods for direct visuo-haptic 4D volume rendering of virtual patient models under respiratory motion. Breathing models are computed based on patient-specific 4D CT image data sequences. Virtual patient models are visualized in real-time by ray casting based rendering of a reference CT image warped by a time-variant displacement field, which is computed using the motion models at run-time. Furthermore, haptic interaction with the animated virtual patient models is provided by using the displacements computed at high rendering rates to translate the position of the haptic device into the space of the reference CT image. This concept is applied to virtual palpation and the haptic simulation of insertion of a virtual bendable needle. To this aim, different motion models that are applicable in real-time are presented and the methods are integrated into a needle puncture training simulation framework, which can be used for simulated biopsy or vessel puncture in the liver. To confirm real-time applicability, a performance analysis of the resulting framework is given. It is shown that the presented methods achieve mean update rates around 2,000 Hz for haptic simulation and interactive frame rates for volume rendering and thus are well suited for visuo-haptic rendering of virtual patients under respiratory motion.

An Image-based Multiproxy Palpation Algorithm for Patient-Specific VR-Simulation.

Palpation is the first step for many medical interventions. To provide an immersive virtual training and planning environment, the palpation step has to be successfully modeled and simulated. Here, we present a multiproxy approach that calculates friction and surface resistance forces for multiple contact points on finger tips or virtual tools like ultrasound probes and displays the resulting force and torque on a 6DOF haptic device. No manual or time intensive segmentation of patient image data is needed to create a simulation based on CT data and thus our approach is usable for patient-specific simulation of palpation.

Ray-casting based evaluation framework for haptic force feedback during percutaneous transhepatic catheter drainage punctures.

PURPOSE: Development of new needle insertion force feedback algorithms requires comparison with a gold standard method. A new evaluation framework was formulated and tested on needle punctures for percutaneous transhepatic catheter drainage (PTCD). METHODS: Needle insertion is an established procedure for minimally invasive interventions in the liver. Up-to-date, needle insertions are precisely planned using 2D axial CT slices from 3D data sets. To provide a 3D virtual reality and haptic training and planning environment, the full segmentation of patient data is often a mandatory step. To lessen the time required for manual segmentation, we propose direct haptic volume-rendering based on CT gray values and partially segmented patient data. The core contribution is a new force output evaluation method driven by a ray-casting technique that defines paths from the skin to target structures, i.e., the right hepatic duct near the juncture with the common hepatic duct. A ray-casting method computes insertion trajectories from the skin to the duct considering no-go structures and plausibility criteria. A rating system scores each trajectory. Finally, the best insertion trajectories are selected that reach the target. Along the selected paths, force output comparison between a reference system and the new haptic force output algorithm is carried out, quantified and visualized. RESULTS: The evaluation framework is presented along with an exemplary study of the liver using the atlas data set from a reference patient. In a comparison of our reference method to a newer algorithm, force outputs are found to be similar in 99% of the paths. CONCLUSION: The proposed evaluation framework allows reliable detection of problematic PTCD trajectories and provides valuable hints to improve force feedback algorithm development.

Optimized image-based soft tissue deformation algorithms for visualization of haptic needle insertion.

Real-time surgical simulation relies on the fast computation of soft tissue deformations. In this paper, we present image-based algorithms for computing the deformations of a volumetric image during a needle insertion in real-time. The algorithms are based on diffusive and linear elastic finite difference methods as utilized in image registration. For an evaluation, the methods are compared to a finite element simulation of the pre-puncture phase of a needle insertion. Furthermore, the methods are improved and tested; contrary to our assumption, an improved diffusion based approach outperforms a linear elastic approach. The algorithms are used to perform a VR simulation of a needle insertion with visuo-haptic feedback.

Direct haptic volume rendering in lumbar puncture simulation.

The preparation phase for surgical simulations often comprises the segmentation of patient data, which is needed for realistic visual and haptic rendering. Expert segmentation of 3D patient data sets can last from several hours to days. In this paper we introduce a direct haptic volume rendering approach for lumbar punctures. Preparation time spent for segmentation is much shorter and compared to our reference system nearly identical force output at the needle tip can be observed. The number of structures to be completely segmented by an expert is reduced from 11 to 3 tissues in abdominal data sets with 300 slices.