Automatic hepatic tumor segmentation in intra-operative ultrasound: a supervised deep-learning approach.

Purpose: Training and evaluation of the performance of a supervised deep-learning model for the segmentation of hepatic tumors from intraoperative US (iUS) images, with the purpose of improving the accuracy of tumor margin assessment during liver surgeries and the detection of lesions during colorectal surgeries. Approach: In this retrospective study, a U-Net network was trained with the nnU-Net framework in different configurations for the segmentation of CRLM from iUS. The model was trained on B-mode intraoperative hepatic US images, hand-labeled by an expert clinician. The model was tested on an independent set of similar images. The average age of the study population was 61.9 ± 9.9 years. Ground truth for the test set was provided by a radiologist, and three extra delineation sets were used for the computation of inter-observer variability. Results: The presented model achieved a DSC of 0.84 (p=0.0037), which is comparable to the expert human raters scores. The model segmented hypoechoic and mixed lesions more accurately (DSC of 0.89 and 0.88, respectively) than hyper- and isoechoic ones (DSC of 0.70 and 0.60, respectively) only missing isoechoic or >20 mm in diameter (8% of the tumors) lesions. The inclusion of extra margins of probable tumor tissue around the lesions in the training ground truth resulted in lower DSCs of 0.75 (p=0.0022). Conclusion: The model can accurately segment hepatic tumors from iUS images and has the potential to speed up the resection margin definition during surgeries and the detection of lesion in screenings by automating iUS assessment.

Ultrasound guidance in navigated liver surgery: toward deep-learning enhanced compensation of deformation and organ motion.

PURPOSE: Accuracy of image-guided liver surgery is challenged by deformation of the liver during the procedure. This study aims at improving navigation accuracy by using intraoperative deep learning segmentation and nonrigid registration of hepatic vasculature from ultrasound (US) images to compensate for changes in liver position and deformation. METHODS: This was a single-center prospective study of patients with liver metastases from any origin. Electromagnetic tracking was used to follow US and liver movement. A preoperative 3D model of the liver, including liver lesions, and hepatic and portal vasculature, was registered with the intraoperative organ position. Hepatic vasculature was segmented using a reduced 3D U-Net and registered to preoperative imaging after initial alignment followed by nonrigid registration. Accuracy was assessed as Euclidean distance between the tumor center imaged in the intraoperative US and the registered preoperative image. RESULTS: Median target registration error (TRE) after initial alignment was 11.6 mm in 25 procedures and improved to 6.9 mm after nonrigid registration (p = 0.0076). The number of TREs above 10 mm halved from 16 to 8 after nonrigid registration. In 9 cases, registration was performed twice after failure of the first attempt. The first registration cycle was completed in median 11 min (8:00-18:45 min) and a second in 5 min (2:30-10:20 min). CONCLUSION: This novel registration workflow using automatic vascular detection and nonrigid registration allows to accurately localize liver lesions. Further automation in the workflow is required in initial alignment and classification accuracy.

Ultrasound-based navigation for open liver surgery using active liver tracking.

PURPOSE: Despite extensive preoperative imaging, intraoperative localization of liver lesions after systemic treatment can be challenging. Therefore, an image-guided navigation setup is explored that links preoperative diagnostic scans and 3D models to intraoperative ultrasound (US), enabling overlay of detailed diagnostic images on intraoperative US. Aim of this study is to assess the workflow and accuracy of such a navigation system which compensates for liver motion. METHODS: Electromagnetic (EM) tracking was used for organ tracking and movement of the transducer. After laparotomy, a sensor was attached to the liver surface while the EM-tracked US transducer enabled image acquisition and landmark digitization. Landmarks surrounding the lesion were selected during patient-specific preoperative 3D planning and identified for registration during surgery. Endpoints were accuracy and additional times of the investigative steps. Accuracy was computed at the center of the target lesion. RESULTS: In total, 22 navigated procedures were performed. Navigation provided useful visualization of preoperative 3D models and their overlay on US imaging. Landmark-based registration resulted in a mean fiducial registration error of 10.3 ± 4.3 mm, and a mean target registration error of 8.5 ± 4.2 mm. Navigation was available after an average of 12.7 min. CONCLUSION: We developed a navigation method combining ultrasound with active liver tracking for organ motion compensation, with an accuracy below 10 mm. Fixation of the liver sensor near the target lesion compensates for local movement and contributes to improved reliability during navigation. This represents an important step forward in providing surgical navigation throughout the procedure. TRIAL REGISTRATION: This study is registered in the Netherlands Trial Register (number NL7951).

Technical note: Validation of 3D ultrasound for image registration during oncological liver surgery.

PURPOSE: Registration of pre- and intraoperative images is a crucial step of surgical liver navigation, where rigid registration of vessel centerlines is currently commonly used. When using 3D ultrasound (US), accuracy during navigation might be influenced by the size of the intraoperative US volume, yet the relationship between registration accuracy and US volume size is understudied. In this study, we specify an optimal 3D US volume size for registration using varying volumes of liver vasculature. While previous studies measured accuracy at registered fiducials, in this work, accuracy is determined at the target lesion which is clinically the most relevant structure. METHODS: Three-dimensional US volumes were acquired in 14 patients after laparotomy and liver mobilization. Manual segmentation of vasculature and centerline extraction was performed. Intraoperative and preoperative vasculature centerlines were registered with coherent point drift, using different sub-volumes (sphere with radius r = 30, 40, …, 120 mm). Accuracy was measured by fiducial registration error (FRE) between vessel centerlines and target registration error (TRE) at the center of the target lesion. RESULTS: The lowest FRE for vessel registration was reached with r = 50 mm (6.5 ± 2.5 mm), the highest with r = 120 mm (7.1 ± 2.1 mm). Clinical accuracy at the target lesion, resulted most accurate (TRE = 8.8 ± 5.0 mm) in sub-volumes with a radius of 50 mm. Smaller US sub-volumes resulted in lower average TREs when compared to larger US sub-volumes (Pearson's correlation coefficient R = 0.91, p < 0.001). CONCLUSION: Our results indicate that there is a linear correlation between US volume size and registration accuracy at the tumor. Volumes with radii of 50 mm around the target lesion yield higher accuracy (p < 0.05) (Trial number IRBd18032, 11 September 2018).

CBCT-based navigation system for open liver surgery: Accurate guidance toward mobile and deformable targets with a semi-rigid organ approximation and electromagnetic tracking of the liver.

PURPOSE: The surgical navigation system that provides guidance throughout the surgery can facilitate safer and more radical liver resections, but such a system should also be able to handle organ motion. This work investigates the accuracy of intraoperative surgical guidance during open liver resection, with a semi-rigid organ approximation and electromagnetic tracking of the target area. METHODS: The suggested navigation technique incorporates a preoperative 3D liver model based on diagnostic 4D MRI scan, intraoperative contrast-enhanced CBCT imaging and electromagnetic (EM) tracking of the liver surface, as well as surgical instruments, by means of six degrees-of-freedom micro-EM sensors. RESULTS: The system was evaluated during surgeries with 35 patients and resulted in an accurate and intuitive real-time visualization of liver anatomy and tumor's location, confirmed by intraoperative checks on visible anatomical landmarks. Based on accuracy measurements verified by intraoperative CBCT, the system's average accuracy was 4.0 ± 3.0 mm, while the total surgical delay due to navigation stayed below 20 min. CONCLUSIONS: The electromagnetic navigation system for open liver surgery developed in this work allows for accurate localization of liver lesions and critical anatomical structures surrounding the resection area, even when the liver was manipulated. However, further clinically integrating the method requires shortening the guidance-related surgical delay, which can be achieved by shifting to faster intraoperative imaging like ultrasound. Our approach is adaptable to navigation on other mobile and deformable organs, and therefore may benefit various clinical applications.

Clinical Implementation of In-House Developed MR-Based Patient-Specific 3D Models of Liver Anatomy.

Knowledge of patient-specific liver anatomy is key to patient safety during major hepatobiliary surgery. Three-dimensional (3D) models of patient-specific liver anatomy based on diagnostic MRI images can provide essential vascular and biliary anatomical insight during surgery. However, a method for generating these is not yet publicly available. This paper describes how these 3D models of the liver can be generated using open source software, and then subsequently integrated into a sterile surgical environment. The most common image quality aspects that degrade the quality of the 3D models as well possible ways of eliminating these are also discussed. Per patient, a single diagnostic multiphase MRI scan with hepatospecific contrast agent was used for automated segmentation of liver contour, arterial, portal, and venous anatomy, and the biliary tree. Subsequently, lesions were delineated manually. The resulting interactive 3D model could be accessed during surgery on a sterile covered tablet. Up to now, such models have been used in 335 surgical procedures. Their use simplified the surgical treatment of patients with a high number of liver metastases and contributed to the localization of vanished lesions in cases of a radiological complete response to neoadjuvant treatment. They facilitated perioperative verification of the relationship of tumors and the surrounding vascular and biliary anatomy, and eased decision-making before and during surgery.