Hybrid optical-vision tracking in laparoscopy: accuracy of navigation and ultrasound reconstruction.

INTRODUCTION: The use of laparoscopic and robotic liver surgery is increasing. However, it presents challenges such as limited field of view and organ deformations. Surgeons rely on laparoscopic ultrasound (LUS) for guidance, but mentally correlating ultrasound images with pre-operative volumes can be difficult. In this direction, surgical navigation systems are being developed to assist with intra-operative understanding. One approach is performing intra-operative ultrasound 3D reconstructions. The accuracy of these reconstructions depends on tracking the LUS probe. MATERIAL AND METHODS: This study evaluates the accuracy of LUS probe tracking and ultrasound 3D reconstruction using a hybrid tracking approach. The LUS probe is tracked from laparoscope images, while an optical tracker tracks the laparoscope. The accuracy of hybrid tracking is compared to full optical tracking using a dual-modality tool. Ultrasound 3D reconstruction accuracy is assessed on an abdominal phantom with CT transformed into the optical tracker's coordinate system. RESULTS: Hybrid tracking achieves a tracking error < 2 mm within 10 cm between the laparoscope and the LUS probe. The ultrasound reconstruction accuracy is approximately 2 mm. CONCLUSION: Hybrid tracking shows promising results that can meet the required navigation accuracy for laparoscopic liver surgery.

Laparoscopic parenchyma-sparing liver resection for large (≥ 50 mm) colorectal metastases.

BACKGROUND: Traditionally, patients with large liver tumors (≥ 50 mm) have been considered for anatomic major hepatectomy. Laparoscopic resection of large liver lesions is technically challenging and often performed by surgeons with extensive experience. The current study aimed to evaluate the surgical and oncologic safety of laparoscopic parenchyma-sparing liver resection in patients with large colorectal metastases. METHODS: Patients who primarily underwent laparoscopic parenchyma-sparing liver resection (less than 3 consecutive liver segments) for colorectal liver metastases between 1999 and 2019 at Oslo University Hospital were analyzed. In some recent cases, a computer-assisted surgical planning system was used to better visualize and understand the patients' liver anatomy, as well as a tool to further improve the resection strategy. The surgical and oncologic outcomes of patients with large (≥ 50 mm) and small (< 50 mm) tumors were compared. Multivariable Cox-regression analysis was performed to identify risk factors for survival. RESULTS: In total 587 patients met the inclusion criteria (large tumor group, n = 59; and small tumor group, n = 528). Median tumor size was 60 mm (range, 50-110) in the large tumor group and 21 mm (3-48) in the small tumor group (p < 0.001). Patient age and CEA level were higher in the large tumor group (8.4 µg/L vs. 4.6 µg/L, p < 0.001). Operation time and conversion rate were similar, while median blood loss was higher in the large tumor group (500 ml vs. 200 ml, p < 0.001). Patients in the large tumor group had shorter 5 year overall survival (34% vs 49%, p = 0.027). However, in the multivariable Cox-regression analysis tumor size did not impact survival, unlike parameters such as age, ASA score, CEA level, extrahepatic disease at liver surgery, and positive lymph nodes in the primary tumor. CONCLUSION: Laparoscopic parenchyma-sparing resections for large colorectal liver metastases provide satisfactory short and long-term outcomes.

Navigated liver surgery: State of the art and future perspectives.

BACKGROUND: In recent years, the development of digital imaging technology has had a significant influence in liver surgery. The ability to obtain a 3-dimensional (3D) visualization of the liver anatomy has provided surgery with virtual reality of simulation 3D computer models, 3D printing models and more recently holograms and augmented reality (when virtual reality knowledge is superimposed onto reality). In addition, the utilization of real-time fluorescent imaging techniques based on indocyanine green (ICG) uptake allows clinicians to precisely delineate the liver anatomy and/or tumors within the parenchyma, applying the knowledge obtained preoperatively through digital imaging. The combination of both has transformed the abstract thinking until now based on 2D imaging into a 3D preoperative conception (virtual reality), enhanced with real-time visualization of the fluorescent liver structures, effectively facilitating intraoperative navigated liver surgery (augmented reality). DATA SOURCES: A literature search was performed from inception until January 2021 in MEDLINE (PubMed), Embase, Cochrane library and database for systematic reviews (CDSR), Google Scholar, and National Institute for Health and Clinical Excellence (NICE) databases. RESULTS: Fifty-one pertinent articles were retrieved and included. The different types of digital imaging technologies and the real-time navigated liver surgery were estimated and compared. CONCLUSIONS: ICG fluorescent imaging techniques can contribute essentially to the real-time definition of liver segments; as a result, precise hepatic resection can be guided by the presence of fluorescence. Furthermore, 3D models can help essentially to further advancing of precision in hepatic surgery by permitting estimation of liver volume and functional liver remnant, delineation of resection lines along the liver segments and evaluation of tumor margins. In liver transplantation and especially in living donor liver transplantation (LDLT), 3D printed models of the donor's liver and models of the recipient's hilar anatomy can contribute further to improving the results. In particular, pediatric LDLT abdominal cavity models can help to manage the largest challenge of this procedure, namely large-for-size syndrome.

Influence of sampling accuracy on augmented reality for laparoscopic image-guided surgery.

PURPOSE: This study aims to evaluate the accuracy of point-based registration (PBR) when used for augmented reality (AR) in laparoscopic liver resection surgery. MATERIAL AND METHODS: The study was conducted in three different scenarios in which the accuracy of sampling targets for PBR decreases: using an assessment phantom with machined divot holes, a patient-specific liver phantom with markers visible in computed tomography (CT) scans and in vivo, relying on the surgeon's anatomical understanding to perform annotations. Target registration error (TRE) and fiducial registration error (FRE) were computed using five randomly selected positions for image-to-patient registration. RESULTS: AR with intra-operative CT scanning showed a mean TRE of 6.9 mm for the machined phantom, 7.9 mm for the patient-specific phantom and 13.4 mm in the in vivo study. CONCLUSIONS: AR showed an increase in both TRE and FRE throughout the experimental studies, proving that AR is not robust to the sampling accuracy of the targets used to compute image-to-patient registration. Moreover, an influence of the size of the volume to be register was observed. Hence, it is advisable to reduce both errors due to annotations and the size of registration volumes, which can cause large errors in AR systems.

Use of mixed reality for surgery planning: Assessment and development workflow.

Meticulous preoperative planning is an important part of any surgery to achieve high levels of precision and avoid complications. Conventional medical 2D images and their corresponding three-dimensional (3D) reconstructions are the main components of an efficient planning system. However, these systems still use flat screens for visualisation of 3D information, thus losing depth information which is crucial for 3D spatial understanding. Currently, cutting-edge mixed reality systems have shown to be a worthy alternative to provide 3D information to clinicians. In this work, we describe development details of the different steps in the workflow for the clinical use of mixed reality, including results from a qualitative user evaluation and clinical use-cases in laparoscopic liver surgery and heart surgery. Our findings indicate a very high general acceptance of mixed reality devices with our applications and they were consistently rated high for device, visualisation and interaction areas in our questionnaire. Furthermore, our clinical use-cases demonstrate that the surgeons perceived the HoloLens to be useful, recommendable to other surgeons and also provided a definitive answer at a multi-disciplinary team meeting.

Use of mixed reality for improved spatial understanding of liver anatomy.

Introduction: In liver surgery, medical images from pre-operative computed tomography and magnetic resonance imaging are the basis for the decision-making process. These images are used in surgery planning and guidance, especially for parenchyma-sparing hepatectomies. Though medical images are commonly visualized in two dimensions (2D), surgeons need to mentally reconstruct this information in three dimensions (3D) for a spatial understanding of the anatomy. The aim of this work is to investigate whether the use of a 3D model visualized in mixed reality with Microsoft HoloLens increases the spatial understanding of the liver, compared to the conventional way of using 2D images.Material and methods: In this study, clinicians had to identify liver segments associated to lesions.Results: Twenty-eight clinicians with varying medical experience were recruited for the study. From a total of 150 lesions, 89 were correctly assigned without significant difference between the modalities. The median time for correct identification was 23.5 [4-138] s using the magnetic resonance imaging images and 6.00 [1-35] s using HoloLens (p < 0.001).Conclusions: The use of 3D liver models in mixed reality significantly decreases the time for tasks requiring a spatial understanding of the organ. This may significantly decrease operating time and improve use of resources.

Laparoscopic liver resection for non-colorectal non-neuroendocrine metastases: perioperative and oncologic outcomes.

BACKGROUND: Liver resection is a treatment of choice for colorectal and neuroendocrine liver metastases, and laparoscopy is an accepted approach for surgical treatment of these patients. The role of liver resection for patients with non-colorectal non-neuroendocrine liver metastases (NCNNLM), however, is still disputable. Outcomes of laparoscopic liver resection for this group of patients have not been analyzed. MATERIAL AND METHODS: In this retrospective study, patients who underwent laparoscopic liver resection for NCNNLM at Oslo University Hospital between April 2000 and January 2018 were analyzed. Perioperative and oncologic data of these patients were examined. Postoperative morbidity was classified using the Accordion classification. Kaplan-Meier method was used for survival analysis. Median follow-up was 26 (IQR, 12-41) months. RESULTS: Fifty-one patients were identified from a prospectively collected database. The histology of primary tumors was classified as adenocarcinoma (n = 16), sarcoma (n = 4), squamous cell carcinoma (n = 4), melanoma (n = 16), gastrointestinal stromal tumor (n = 9), and adrenocortical carcinoma (n = 2). The median operative time was 147 (IQR, 95-225) min, while the median blood loss was 200 (IQR, 50-500) ml. Nine (18%) patients experienced postoperative complications. There was no 90-day mortality in this study. Thirty-five (68%) patients developed disease recurrence or progression. Seven (14%) patients underwent repeat surgical procedure for recurrent liver metastases. One-, three-, and five-year overall survival rates were 85%, 52%, and 38%, respectively. The median overall survival was 37 (95%CI, 25 to 49) months. CONCLUSION: Laparoscopic liver resection for NCNNLM results in good outcomes and should be considered in patients selected for surgical treatment.

Laparoscopic Parenchyma-sparing Liver Resection for Colorectal Metastases.

BACKGROUND: Laparoscopic liver resection (LLR) of colorectal liver metastases (CLM) is increasingly performed in specialized centers. While there is a trend towards a parenchyma-sparing strategy in multimodal treatment for CLM, its role is yet unclear. In this study we present short- and long-term outcomes of laparoscopic parenchyma-sparing liver resection (LPSLR) at a single center. PATIENTS AND METHODS: LLR were performed in 951 procedures between August 1998 and March 2017 at Oslo University Hospital, Oslo, Norway. Patients who primarily underwent LPSLR for CLM were included in the study. LPSLR was defined as non-anatomic hence the patients who underwent hemihepatectomy and sectionectomy were excluded. Perioperative and oncologic outcomes were analyzed. The Accordion classification was used to grade postoperative complications. The median follow-up was 40 months. RESULTS: 296 patients underwent primary LPSLR for CLM. A single specimen was resected in 204 cases, multiple resections were performed in 92 cases. 5 laparoscopic operations were converted to open. The median operative time was 134 minutes, blood loss was 200 ml and hospital stay was 3 days. There was no 90-day mortality in this study. The postoperative complication rate was 14.5%. 189 patients developed disease recurrence. Recurrence in the liver occurred in 146 patients (49%), of whom 85 patients underwent repeated surgical treatment (liver resection [n = 69], ablation [n = 14] and liver transplantation [n = 2]). Five-year overall survival was 48%, median overall survival was 56 months. CONCLUSIONS: LPSLR of CLM can be performed safely with the good surgical and oncological results. The technique facilitates repeated surgical treatment, which may improve survival for patients with CLM.

Mixed reality - new image technology in experimental use. / Blandet virkelighet ­ ny bildeteknologi i klinisk utprøving.

New methods for holographic visualisation provide a true three-dimensional experience of medical images. The technique is generating great interest among surgeons.

Learning deep abdominal CT registration through adaptive loss weighting and synthetic data generation.

PURPOSE: This study aims to explore training strategies to improve convolutional neural network-based image-to-image deformable registration for abdominal imaging. METHODS: Different training strategies, loss functions, and transfer learning schemes were considered. Furthermore, an augmentation layer which generates artificial training image pairs on-the-fly was proposed, in addition to a loss layer that enables dynamic loss weighting. RESULTS: Guiding registration using segmentations in the training step proved beneficial for deep-learning-based image registration. Finetuning the pretrained model from the brain MRI dataset to the abdominal CT dataset further improved performance on the latter application, removing the need for a large dataset to yield satisfactory performance. Dynamic loss weighting also marginally improved performance, all without impacting inference runtime. CONCLUSION: Using simple concepts, we improved the performance of a commonly used deep image registration architecture, VoxelMorph. In future work, our framework, DDMR, should be validated on different datasets to further assess its value.

Evaluation of clinical applicability of automated liver parenchyma segmentation of multi-center magnetic resonance images.

Purpose: Automated algorithms for liver parenchyma segmentation can be used to create patient-specific models (PSM) that assist clinicians in surgery planning. In this work, we analyze the clinical applicability of automated deep learning methods together with level set post-processing for liver segmentation in contrast-enhanced T1-weighted magnetic resonance images. Methods: UNet variants with/without attention gate, multiple loss functions, and level set post-processing were used in the workflow. A multi-center, multi-vendor dataset from Oslo laparoscopic versus open liver resection for colorectal liver metastasis clinical trial is used in our study. The dataset of 150 volumes is divided as 81:25:25:19 corresponding to train:validation:test:clinical evaluation respectively. We evaluate the clinical use, time to edit automated segmentation, tumor regions, boundary leakage, and over-and-under segmentations of predictions. Results: The deep learning algorithm shows a mean Dice score of 0.9696 in liver segmentation, and we also examined the potential of post-processing to improve the PSMs. The time to create clinical use segmentations of level set post-processed predictions shows a median time of 16 min which is 2 min less than deep learning inferences. The intra-observer variations between manually corrected deep learning and level set post-processed segmentations show a 3% variation in the Dice score. The clinical evaluation shows that 7 out of 19 cases of both deep learning and level set post-processed segmentations contain all required anatomy and pathology, and hence these results could be used without any manual corrections. Conclusions: The level set post-processing reduces the time to create clinical standard segmentations, and over-and-under segmentations to a certain extent. The time advantage greatly supports clinicians to spend their valuable time with patients.

Evaluation of a novel navigation platform for laparoscopic liver surgery with organ deformation compensation using injected fiducials.

In laparoscopic liver resection, surgeons conventionally rely on anatomical landmarks detected through a laparoscope, preoperative volumetric images and laparoscopic ultrasound to compensate for the challenges of minimally invasive access. Image guidance using optical tracking and registration procedures is a promising tool, although often undermined by its inaccuracy. This study evaluates a novel surgical navigation solution that can compensate for liver deformations using an accurate and effective registration method. The proposed solution relies on a robotic C-arm to perform registration to preoperative CT/MRI image data and allows for intraoperative updates during resection using fluoroscopic images. Navigation is offered both as a 3D liver model with real-time instrument visualization, as well as an augmented reality overlay on the laparoscope camera view. Testing was conducted through a pre-clinical trial which included four porcine models. Accuracy of the navigation system was measured through two evaluation methods: liver surface fiducials reprojection and a comparison between planned and navigated resection margins. Target Registration Error with the fiducials evaluation shows that the accuracy in the vicinity of the lesion was 3.78±1.89 mm. Resection margin evaluations resulted in an overall median accuracy of 4.44 mm with a maximum error of 9.75 mm over the four subjects. The presented solution is accurate enough to be potentially clinically beneficial for surgical guidance in laparoscopic liver surgery.

GPU acceleration of liver enhancement for tumor segmentation.

BACKGROUND AND OBJECTIVE: Medical image segmentation plays a vital role in medical image analysis. There are many algorithms developed for medical image segmentation which are based on edge or region characteristics. These are dependent on the quality of the image. The contrast of a CT or MRI image plays an important role in identifying region of interest i.e. lesion(s). In order to enhance the contrast of image, clinicians generally use manual histogram adjustment technique which is based on 1D histogram specification. This is time consuming and results in poor distribution of pixels over the image. Cross modality based contrast enhancement is 2D histogram specification technique. This is robust and provides a more uniform distribution of pixels over CT image by exploiting the inner structure information from MRI image. This helps in increasing the sensitivity and accuracy of lesion segmentation from enhanced CT image. The sequential implementation of cross modality based contrast enhancement is slow. Hence we propose GPU acceleration of cross modality based contrast enhancement for tumor segmentation. METHODS: The aim of this study is fast parallel cross modality based contrast enhancement for CT liver images. This includes pairwise 2D histogram, histogram equalization and histogram matching. The sequential implementation of the cross modality based contrast enhancement is computationally expensive and hence time consuming. We propose persistence and grid-stride loop based fast parallel contrast enhancement for CT liver images. We use enhanced CT liver image for the lesion or tumor segmentation. We implement the fast parallel gradient based dynamic seeded region growing for lesion segmentation. RESULTS: The proposed parallel approach is 104.416 ( ±  5.166) times faster compared to the sequential implementation and increases the sensitivity and specificity of tumor segmentation. CONCLUSION: The cross modality approach is inspired by 2D histogram specification which incorporates spatial information existing in both guidance and input images for remapping the input image intensity values. The cross modality based liver contrast enhancement improves the quality of tumor segmentation.

The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery.

Conventional surgical navigation systems rely on preoperative imaging to provide guidance. In laparoscopic liver surgery, insufflation of the abdomen (pneumoperitoneum) can cause deformations on the liver, introducing inaccuracies in the correspondence between the preoperative images and the intraoperative reality. This study evaluates the improvements provided by intraoperative imaging for laparoscopic liver surgical navigation, when displayed as augmented reality (AR). Significant differences were found in terms of accuracy of the AR, in favor of intraoperative imaging. In addition, results showed an effect of user-induced error: image-to-patient registration based on annotations performed by clinicians caused 33% more inaccuracy as compared to image-to-patient registration algorithms that do not depend on user annotations. Hence, to achieve accurate surgical navigation for laparoscopic liver surgery, intraoperative imaging is recommendable to compensate for deformation. Moreover, user annotation errors may lead to inaccuracies in registration processes.