Comparing a virtual reality head-mounted display to on-screen three-dimensional visualization and two-dimensional computed tomography data for training in decision making in hepatic surgery: a randomized controlled study.

OBJECTIVE: Evaluation of the benefits of a virtual reality (VR) environment with a head-mounted display (HMD) for decision-making in liver surgery. BACKGROUND: Training in liver surgery involves appraising radiologic images and considering the patient's clinical information. Accurate assessment of 2D-tomography images is complex and requires considerable experience, and often the images are divorced from the clinical information. We present a comprehensive and interactive tool for visualizing operation planning data in a VR environment using a head-mounted-display and compare it to 3D visualization and 2D-tomography. METHODS: Ninety medical students were randomized into three groups (1:1:1 ratio). All participants analyzed three liver surgery patient cases with increasing difficulty. The cases were analyzed using 2D-tomography data (group "2D"), a 3D visualization on a 2D display (group "3D") or within a VR environment (group "VR"). The VR environment was displayed using the "Oculus Rift ™" HMD technology. Participants answered 11 questions on anatomy, tumor involvement and surgical decision-making and 18 evaluative questions (Likert scale). RESULTS: Sum of correct answers were significantly higher in the 3D (7.1 ± 1.4, p < 0.001) and VR (7.1 ± 1.4, p < 0.001) groups than the 2D group (5.4 ± 1.4) while there was no difference between 3D and VR (p = 0.987). Times to answer in the 3D (6:44 ± 02:22 min, p < 0.001) and VR (6:24 ± 02:43 min, p < 0.001) groups were significantly faster than the 2D group (09:13 ± 03:10 min) while there was no difference between 3D and VR (p = 0.419). The VR environment was evaluated as most useful for identification of anatomic anomalies, risk and target structures and for the transfer of anatomical and pathological information to the intraoperative situation in the questionnaire. CONCLUSIONS: A VR environment with 3D visualization using a HMD is useful as a surgical training tool to accurately and quickly determine liver anatomy and tumor involvement in surgery.

Intraoperative liver deformation and organ motion caused by ventilation, laparotomy, and pneumoperitoneum in a porcine model for image-guided liver surgery.

BACKGROUND: Image-guidance promises to make complex situations in liver interventions safer. Clinical success is limited by intraoperative organ motion due to ventilation and surgical manipulation. The aim was to assess influence of different ventilatory and operative states on liver motion in an experimental model. METHODS: Liver motion due to ventilation (expiration, middle, and full inspiration) and operative state (native, laparotomy, and pneumoperitoneum) was assessed in a live porcine model (n = 10). Computed tomography (CT)-scans were taken for each pig for each possible combination of factors. Liver motion was measured by the vectors between predefined landmarks along the hepatic vein tree between CT scans after image segmentation. RESULTS: Liver position changed significantly with ventilation. Peripheral regions of the liver showed significantly higher motion (maximal Euclidean motion 17.9 ± 2.7 mm) than central regions (maximal Euclidean motion 12.6 ± 2.1 mm, p < 0.001) across all operative states. The total average motion measured 11.6 ± 0.7 mm (p < 0.001). Between the operative states, the position of the liver changed the most from native state to pneumoperitoneum (14.6 ± 0.9 mm, p < 0.001). From native state to laparotomy comparatively, the displacement averaged 9.8 ± 1.2 mm (p < 0.001). With pneumoperitoneum, the breath-dependent liver motion was significantly reduced when compared to other modalities. Liver motion due to ventilation was 7.7 ± 0.6 mm during pneumoperitoneum, 13.9 ± 1.1 mm with laparotomy, and 13.5 ± 1.4 mm in the native state (p < 0.001 in all cases). CONCLUSIONS: Ventilation and application of pneumoperitoneum caused significant changes in liver position. Liver motion was reduced but clearly measurable during pneumoperitoneum. Intraoperative guidance/navigation systems should therefore account for ventilation and intraoperative changes of liver position and peripheral deformation.

[Splenic surgery in hematological diseases : Indications and surgical technique]. / Chirurgie der Milz bei hämatologischen Erkrankungen : Indikationen und operative Technik.

BACKGROUND: Splenic surgery in hematological disorders requires a well-weighted decision on the indications because the medical treatment has rapidly changed in recent years due to new pharmaceutical approaches. OBJECTIVE: Summary of the indications, surgical procedures and perioperative management regarding operative interventions on the spleen in hematological disorders. MATERIAL AND METHODS: Selective literature search and summary of reviews and guideline recommendations. RESULTS: In hematological disorders surgical procedures of the spleen (splenectomy and partial splenectomy) are an important part of the repertoire in the treatment. In recent years the indications for surgery have become narrower because of new forms of medicinal treatment. Especially in hereditary spherocytosis, immune thrombocytopenia and symptomatic splenomegaly and hypersplenism it is still of importance. The minimally invasive splenectomy is regarded as the gold standard. The spleen has an important immune and sequestration function, therefore preoperative and postoperative infectious and thromboembolic events have to be anticipated and prevented. A close interdisciplinary cooperation with hematologists is essential for an optimal outcome of patients. CONCLUSION: The minimally invasive splenectomy and partial splenectomy are part of the surgical repertoire in the diagnostics and treatment of hematological disorders. Because of novel medicinal approaches the therapeutic protocols are continuously changing. A close cooperation with hematologists is important for the optimal evaluation of the indications and the perioperative management.

IMHOTEP: cross-professional evaluation of a three-dimensional virtual reality system for interactive surgical operation planning, tumor board discussion and immersive training for complex liver surgery in a head-mounted display.

BACKGROUND: Virtual reality (VR) with head-mounted displays (HMD) may improve medical training and patient care by improving display and integration of different types of information. The aim of this study was to evaluate among different healthcare professions the potential of an interactive and immersive VR environment for liver surgery that integrates all relevant patient data from different sources needed for planning and training of procedures. METHODS: 3D-models of the liver, other abdominal organs, vessels, and tumors of a sample patient with multiple hepatic masses were created. 3D-models, clinical patient data, and other imaging data were visualized in a dedicated VR environment with an HMD (IMHOTEP). Users could interact with the data using head movements and a computer mouse. Structures of interest could be selected and viewed individually or grouped. IMHOTEP was evaluated in the context of preoperative planning and training of liver surgery and for the potential of broader surgical application. A standardized questionnaire was voluntarily answered by four groups (students, nurses, resident and attending surgeons). RESULTS: In the evaluation by 158 participants (57 medical students, 35 resident surgeons, 13 attending surgeons and 53 nurses), 89.9% found the VR system agreeable to work with. Participants generally agreed that complex cases in particular could be assessed better (94.3%) and faster (84.8%) with VR than with traditional 2D display methods. The highest potential was seen in student training (87.3%), resident training (84.6%), and clinical routine use (80.3%). Least potential was seen in nursing training (54.8%). CONCLUSIONS: The present study demonstrates that using VR with HMD to integrate all available patient data for the preoperative planning of hepatic resections is a viable concept. VR with HMD promises great potential to improve medical training and operation planning and thereby to achieve improvement in patient care.

Effects of laparoscopy, laparotomy, and respiratory phase on liver volume in a live porcine model for liver resection.

BACKGROUND: Hepatectomy, living donor liver transplantations and other major hepatic interventions rely on precise calculation of the total, remnant and graft liver volume. However, liver volume might differ between the pre- and intraoperative situation. To model liver volume changes and develop and validate such pre- and intraoperative assistance systems, exact information about the influence of lung ventilation and intraoperative surgical state on liver volume is essential. METHODS: This study assessed the effects of respiratory phase, pneumoperitoneum for laparoscopy, and laparotomy on liver volume in a live porcine model. Nine CT scans were conducted per pig (N = 10), each for all possible combinations of the three operative (native, pneumoperitoneum and laparotomy) and respiratory states (expiration, middle inspiration and deep inspiration). Manual segmentations of the liver were generated and converted to a mesh model, and the corresponding liver volumes were calculated. RESULTS: With pneumoperitoneum the liver volume decreased on average by 13.2% (112.7 ml ± 63.8 ml, p < 0.0001) and after laparotomy by 7.3% (62.0 ml ± 65.7 ml, p = 0.0001) compared to native state. From expiration to middle inspiration the liver volume increased on average by 4.1% (31.1 ml ± 55.8 ml, p = 0.166) and from expiration to deep inspiration by 7.2% (54.7 ml ± 51.8 ml, p = 0.007). CONCLUSIONS: Considerable changes in liver volume change were caused by pneumoperitoneum, laparotomy and respiration. These findings provide knowledge for the refinement of available preoperative simulation and operation planning and help to adjust preoperative imaging parameters to best suit the intraoperative situation.

One or two trainees per workplace for laparoscopic surgery training courses: results from a randomized controlled trial.

BACKGROUND: There are no standards for optimal utilization of workplaces in laparoscopic training. This study aimed to define whether laparoscopy training should be done alone or in pairs (known as dyad training). METHODS: This was a three-arm randomized controlled trial with laparoscopically naïve medical students (n = 100). Intervention groups participated alone (n = 40) or as dyad (n = 40) in a multimodality training curriculum with e-learning, basic, and procedural skills training using box and VR trainers. The control group (n = 20) had no training. Post-performance of a cadaveric porcine laparoscopic cholecystectomy (LC) was measured as the primary outcome by blinded raters using the objective structured assessment of technical skills (OSATS). Global operative assessment of laparoscopic skills (GOALS), time for LC, and VR performances were secondary outcomes. RESULTS: There were no differences between groups for performance scores [OSATS: alone (40.2 ± 9.8) vs. dyad (39.8 ± 8.6), p = 0.995; alone vs. control (37.1 ± 7.4), p = 0.548; or dyad vs. control, p = 0.590; and GOALS score: alone (10.6 ± 3.0) vs. dyad (10.0 ± 2.7), p = 0.599; alone vs. control (10.1 ± 3.0), p = 0.748; or dyad vs. control, p = 0.998]. Dyad finished LC faster than control [median = 62.5 min (CI 58.0-73.0) vs. 76.5 min (CI 72.0-80+); p = 0.042], while there were no inter-group differences between alone vs. control [median = 69.0 min (CI 62.0-76.0) vs. control; p = 0.099] or alone vs. dyad (p = 0.840). Dyad and alone showed superior performance on the VR trainer vs. control for time, number of movements, and path length, but not for complications and application of cautery. CONCLUSIONS: The curriculum provided trainees with the laparoscopic skills needed to perform LC safely, irrespective of the number of trainees per workplace. Dyad training reduced the operation time needed for LC. Therefore, dyad training seems to be a promising alternative, especially if training time is limited and resources must be used as efficiently as possible. Trial registration German Clinical Trials Register: DRKS00004675.

Mobile, real-time, and point-of-care augmented reality is robust, accurate, and feasible: a prospective pilot study.

BACKGROUND: Augmented reality (AR) systems are currently being explored by a broad spectrum of industries, mainly for improving point-of-care access to data and images. Especially in surgery and especially for timely decisions in emergency cases, a fast and comprehensive access to images at the patient bedside is mandatory. Currently, imaging data are accessed at a distance from the patient both in time and space, i.e., at a specific workstation. Mobile technology and 3-dimensional (3D) visualization of radiological imaging data promise to overcome these restrictions by making bedside AR feasible. METHODS: In this project, AR was realized in a surgical setting by fusing a 3D-representation of structures of interest with live camera images on a tablet computer using marker-based registration. The intent of this study was to focus on a thorough evaluation of AR. Feasibility, robustness, and accuracy were thus evaluated consecutively in a phantom model and a porcine model. Additionally feasibility was evaluated in one male volunteer. RESULTS: In the phantom model (n = 10), AR visualization was feasible in 84% of the visualization space with high accuracy (mean reprojection error ± standard deviation (SD): 2.8 ± 2.7 mm; 95th percentile = 6.7 mm). In a porcine model (n = 5), AR visualization was feasible in 79% with high accuracy (mean reprojection error ± SD: 3.5 ± 3.0 mm; 95th percentile = 9.5 mm). Furthermore, AR was successfully used and proved feasible within a male volunteer. CONCLUSIONS: Mobile, real-time, and point-of-care AR for clinical purposes proved feasible, robust, and accurate in the phantom, animal, and single-trial human model shown in this study. Consequently, AR following similar implementation proved robust and accurate enough to be evaluated in clinical trials assessing accuracy, robustness in clinical reality, as well as integration into the clinical workflow. If these further studies prove successful, AR might revolutionize data access at patient bedside.

IMHOTEP: virtual reality framework for surgical applications.

PURPOSE: The data which is available to surgeons before, during and after surgery is steadily increasing in quantity as well as diversity. When planning a patient's treatment, this large amount of information can be difficult to interpret. To aid in processing the information, new methods need to be found to present multimodal patient data, ideally combining textual, imagery, temporal and 3D data in a holistic and context-aware system. METHODS: We present an open-source framework which allows handling of patient data in a virtual reality (VR) environment. By using VR technology, the workspace available to the surgeon is maximized and 3D patient data is rendered in stereo, which increases depth perception. The framework organizes the data into workspaces and contains tools which allow users to control, manipulate and enhance the data. Due to the framework's modular design, it can easily be adapted and extended for various clinical applications. RESULTS: The framework was evaluated by clinical personnel (77 participants). The majority of the group stated that a complex surgical situation is easier to comprehend by using the framework, and that it is very well suited for education. Furthermore, the application to various clinical scenarios-including the simulation of excitation propagation in the human atrium-demonstrated the framework's adaptability. As a feasibility study, the framework was used during the planning phase of the surgical removal of a large central carcinoma from a patient's liver. CONCLUSION: The clinical evaluation showed a large potential and high acceptance for the VR environment in a medical context. The various applications confirmed that the framework is easily extended and can be used in real-time simulation as well as for the manipulation of complex anatomical structures.

Paradigm shift: cognitive surgery.

In the last hundred years surgery has experienced a dramatic increase of scientific knowledge and innovation. The need to consider best available evidence and to apply technical innovations, such as minimally invasive approaches, challenges the surgeon both intellectually and manually. In order to overcome this challenge, computer scientists and surgeons within the interdisciplinary field of "cognitive surgery" explore and innovate new ways of data processing and management. This article gives a general overview of the topic and outlines selected pre-, intra- and postoperative applications. It explores the possibilities of new intelligent devices and software across the entire treatment process of patients ending in the consideration of an "Intelligent Hospital" or "Hospital 4.0", in which the borders between IT infrastructures, medical devices, medical personnel and patients are bridged by technology. Thereby, the "Hospital 4.0" is an intelligent system, which gives the right information, at the right time, at the right place to the individual stakeholder and thereby helps to decrease complications and improve clinical processes as well as patient outcome.

Study protocol for a randomized controlled trial on a multimodal training curriculum for laparoscopic cholecystectomy - LapTrain.

BACKGROUND: Although minimally invasive surgery (MIS) has replaced many open procedures in visceral surgery, technical and psychomotor obstacles remain a constant challenge for surgeons and trainees. However, there are various training curricula enabling surgeons to acquire the visuospatial and psychomotor abilities additionally required when performing MIS. Currently accepted training modalities include box-trainers, organ and animal models as well as completely simulated training environments, realized in virtual reality (VR) trainers. All of these methods facilitate an adequate training prior to patient contact, so patient safety can benefit as well. This study aims to evaluate the benefit of a structured multi-modality laparoscopy training curriculum. METHODS: Junior and senior surgical residents are included (n = 60). Groups are stratified with concern to previous experience and training of participants. The training curriculum consists of a standardized sequence of available modalities and exercises on box- and VR-trainers. Specific consideration applies to the training effect during the repeated performance of a laparoscopic cholecystectomy (LC) between intervention (training in between LCs) and control group (no training in between LCs). Analysis of training effects is performed using a cadaveric model for LC and objectified using the validated scoring system Global Operative Assessment of Laparoscopic Skills (GOALS). DISCUSSION: This study assesses the value of a multimodal training platform in medical education and postgraduate training and aims at illustrating possible guidelines when establishing such a curriculum. Possible factors of influence, such as varying backgrounds, learning motivation and -success among participants are explored in the data analysis and add beneficially to further evaluating the efficacy of such training to more heterogeneous participant groups like medical students and other professionals.

Direct Observation versus Endoscopic Video Recording-Based Rating with the Objective Structured Assessment of Technical Skills for Training of Laparoscopic Cholecystectomy.

PURPOSE: The validated Objective Structured Assessment of Technical Skills (OSATS) score is used for evaluating laparoscopic surgical performance. It consists of two subscores, a Global Rating Scale (GRS) and a Specific Technical Skills (STS) scale. The OSATS has accepted construct validity for direct observation ratings by experts to discriminate between trainees' levels of experience. Expert time is scarce. Endoscopic video recordings would facilitate assessment with the OSATS. We aimed to compare video OSATS with direct OSATS. METHODS: We included 79 participants with different levels of experience [58 medical students, 15 junior residents (novices), and 6 experts]. Performance of a cadaveric porcine laparoscopic cholecystectomy (LC) was evaluated with OSATS by blinded expert raters by direct observation and then as an endoscopic video recording. Operative time was recorded. RESULTS: Direct OSATS rating and video OSATS rating correlated significantly (x03C1; = 0.33, p = 0.005). Significant construct validity was found for direct OSATS in distinguishing between students or novices and experts. Students and novices were not different in direct OSATS or video OSATS. Mean operative times varied for students (73.4 ± 9.0 min), novices (65.2 ± 22.3 min), and experts (46.8 ± 19.9 min). Internal consistency was high between the GRS and STS subscores for both direct and video OSATS with Cronbach's α of 0.76 and 0.86, respectively. Video OSATS and operative time in combination was a better predictor of direct OSATS than each single parameter. CONCLUSION: Direct OSATS rating was better than endoscopic video rating for differentiating between students or novices and experts for LC and should remain the standard approach for the discrimination of experience levels. However, in the absence of experts for direct rating, video OSATS supplemented with operative time should be used instead of single parameters for predicting direct OSATS scores.

Intraoperative on-the-fly organ-mosaicking for laparoscopic surgery.

The goal of computer-assisted surgery is to provide the surgeon with guidance during an intervention, e.g., using augmented reality. To display preoperative data, soft tissue deformations that occur during surgery have to be taken into consideration. Laparoscopic sensors, such as stereo endoscopes, can be used to create a three-dimensional reconstruction of stereo frames for registration. Due to the small field of view and the homogeneous structure of tissue, reconstructing just one frame, in general, will not provide enough detail to register preoperative data, since every frame only contains a part of an organ surface. A correct assignment to the preoperative model is possible only if the patch geometry can be unambiguously matched to a part of the preoperative surface. We propose and evaluate a system that combines multiple smaller reconstructions from different viewpoints to segment and reconstruct a large model of an organ. Using graphics processing unit-based methods, we achieved four frames per second. We evaluated the system with in silico, phantom, ex vivo, and in vivo (porcine) data, using different methods for estimating the camera pose (optical tracking, iterative closest point, and a combination). The results indicate that the proposed method is promising for on-the-fly organ reconstruction and registration.

Crowdtruth validation: a new paradigm for validating algorithms that rely on image correspondences.

PURPOSE: Feature tracking and 3D surface reconstruction are key enabling techniques to computer-assisted minimally invasive surgery. One of the major bottlenecks related to training and validation of new algorithms is the lack of large amounts of annotated images that fully capture the wide range of anatomical/scene variance in clinical practice. To address this issue, we propose a novel approach to obtaining large numbers of high-quality reference image annotations at low cost in an extremely short period of time. METHODS: The concept is based on outsourcing the correspondence search to a crowd of anonymous users from an online community (crowdsourcing) and comprises four stages: (1) feature detection, (2) correspondence search via crowdsourcing, (3) merging multiple annotations per feature by fitting Gaussian finite mixture models, (4) outlier removal using the result of the clustering as input for a second annotation task. RESULTS: On average, 10,000 annotations were obtained within 24 h at a cost of $100. The annotation of the crowd after clustering and before outlier removal was of expert quality with a median distance of about 1 pixel to a publically available reference annotation. The threshold for the outlier removal task directly determines the maximum annotation error, but also the number of points removed. CONCLUSIONS: Our concept is a novel and effective method for fast, low-cost and highly accurate correspondence generation that could be adapted to various other applications related to large-scale data annotation in medical image computing and computer-assisted interventions.

Crowdsourcing for reference correspondence generation in endoscopic images.

Computer-assisted minimally-invasive surgery (MIS) is often based on algorithms that require establishing correspondences between endoscopic images. However, reference annotations frequently required to train or validate a method are extremely difficult to obtain because they are typically made by a medical expert with very limited resources, and publicly available data sets are still far too small to capture the wide range of anatomical/scene variance. Crowdsourcing is a new trend that is based on outsourcing cognitive tasks to many anonymous untrained individuals from an online community. To our knowledge, this paper is the first to investigate the concept of crowdsourcing in the context of endoscopic video image annotation for computer-assisted MIS. According to our study on publicly available in vivo data with manual reference annotations, anonymous non-experts obtain a median annotation error of 2 px (n = 10,000). By applying cluster analysis to multiple annotations per correspondence, this error can be reduced to about 1 px, which is comparable to that obtained by medical experts (n = 500). We conclude that crowdsourcing is a viable method for generating high quality reference correspondences in endoscopic video images.